Python

Pure Python Paged Attention Engine with LRU KV‑Cache Eviction

A step‑by‑step guide to building a pure Python paged attention engine with LRU cache eviction, perfect for demonstrating systems‑level skills to ML recruiters.

Python code on a dark terminal, visualizing paged KV cache buckets

Build a Paged Attention KV Cache with H2O Eviction in Pure Python

Learn how to implement a paged attention KV cache with H2O eviction in pure Python, a concrete project that demonstrates low‑latency data structures, eviction policies, and production‑ready patterns for AI/ML infrastructure.

Illustration of a Python neural network pipeline

From‑Scratch Speculative Decoding Engine in Pure Python

A hands‑on guide to building a from‑scratch speculative decoding engine in pure Python, with draft‑then‑verify sampling, temperature, top‑k and nucleus (top‑p) sampling, all using only numpy. Perfect for a CV‑boosting side project.

Lazy Loading LLM Weights with Safetensors, Async Prefetch, and Checksum Verification

A practical guide to building a lazy weight loader for large LLMs using safetensors, async prefetch, and checksum verification.

Pure Python GGUF Weight Loader: Streaming Quantized Tensors via Memory-Mapped Files

A step‑by‑step guide to building a pure Python GGUF weight loader that streams quantized tensors using memory‑mapped files, complete with code, testing, and production‑ready extensions.