A hands-on guide to building a pure-Python paged attention KV-cache with LRU and frequency-based eviction policies — a portfolio project that demonstrates deep systems engineering skills.
Step-by-step build of a real, runnable paged KV cache manager in Python: blocks, block tables, prefix sharing via a radix tree, copy-on-write for beam search, plus a roadmap to senior-level extensions.
Ship a runnable paged-attention KV cache manager that demonstrates real LLM systems skill: block tables, prefix-sharing across requests, and eviction policies that actually move the needle on memory.
Build a from-scratch paged KV cache manager for LLM inference, with block-level prefix sharing and copy-on-write semantics. A real systems project for your CV.
TL;DR — A paged KV cache with grouped-query attention (GQA) sharing and a prefix-reuse eviction policy is one of the most credible LLM-systems projects you can ship to a CV in a weekend. It mirrors the design choices in vLLM, SGLang, and the RadixAttention paper, demonstrates that you understand memory hierarchies, attention math, and eviction, and produces a runnable artifact you can benchmark on real prompts. Why This Project Stands Out on a CV Hiring managers for inference, ML platform, and applied-LLM roles scan portfolios for evidence that a candidate can translate a paper into a working system. A paged KV cache hits that target harder than almost any other weekend project because the design space is genuinely contested in production: vLLM ships PagedAttention, SGLang ships RadixAttention, TensorRT-LLM ships in-flight batching with custom paged kernels, and HuggingFace TGI ships its own block manager. Building one from scratch proves you have read those systems and understood why they look the way they do. ...