From-Scratch Safetensors & GGUF Weight Loader with Lazy Memory-Mapped Tensor Streaming and Q4_0/Q8_0 Dequantization in Pure Python
A step‑by‑step guide to creating a lightweight, lazy‑evaluated weight loader that streams safetensors and GGUF tensors, applies Q4_0/Q8_0 dequantization, and can be dropped into any inference pipeline.
Hands-On Build Guide: Tree‑Based Speculative Decoding Engine with Medusa‑Style Draft Heads
A hands‑on implementation of tree‑based speculative decoding with Medusa draft heads in pure Python, ready to run and extend.
Building a Self-Contained Sampling and Decoding Engine in Pure Python
Implement temperature, top‑k, and top‑p sampling from scratch in pure Python, then extend the engine for production use.
Mastering QUIC for Real-Time Voice Pipelines: Datagram Transport in Embedded Networks
A practical guide to deploying QUIC for real-time voice pipelines on embedded devices focusing on latency reliability and connection resilience in constrained networks
From-Scratch Paged Attention Allocator: LRU-Blocked KV-Cache Recompute for LLM Inference in Pure Python
A practical, runnable Python implementation of paged attention with LRU eviction and KV-cache recompute, perfect for LLM inference side projects and hiring manager impressions.