Llm-Inference

Build a Paged KV Cache With Continuous Batching and Prefix Sharing: A From-Scratch Guide

A working engineer’s walkthrough of building a paged KV cache with continuous batching and prefix sharing from scratch — the same ideas vLLM ships in production. Includes runnable Python, an architecture breakdown, tests, and a concrete extension roadmap.

Diagram of a GGUF file layout streaming into memory-mapped safetensors buffers.

Building a GGUF Parser from Scratch: A Portfolio Project That Signals Real Systems Skill

Walk through designing and implementing a GGUF parser that streams tensor shards into memory-mapped safetensors-compatible buffers — a CV-grade systems project with concrete code, tests, and a roadmap to senior-level upgrades.

Diagram of a speculative decoding loop with an n-gram draft model feeding into a target verification step.

Building a Speculative Decoding Engine From Scratch: A Portfolio Project That Actually Signals Systems Skill

Build a runnable speculative decoding engine in Python with an n-gram draft model and target-model verification loop — a portfolio project that demonstrates LLM inference, KV-cache reasoning, and acceptance sampling.

Diagram of paged-attention KV cache blocks mapped across sequences

Build a Paged-Attention KV Cache Manager: A Hands-On Project That Actually Impresses Hiring Managers

Ship a runnable paged-attention KV cache manager that demonstrates real LLM systems skill: block tables, prefix-sharing across requests, and eviction policies that actually move the needle on memory.

Block diagram of paged KV cache with prefix sharing

Building a Paged KV Cache Manager with Prefix Sharing and Copy-on-Write

Build a from-scratch paged KV cache manager for LLM inference, with block-level prefix sharing and copy-on-write semantics. A real systems project for your CV.