Kv-Cache

A diagram of a transformer attention block with sliding window

Build a Fused Causal Self-Attention Block with RoPE and Sliding-Window KV Cache in Pure Python

This guide walks through implementing a fused causal self-attention block with RoPE and a sliding-window KV cache in pure Python, giving you a concrete portfolio piece that highlights systems engineering expertise.

Diagram of a paged KV cache with blocks, block tables, and a prefix-sharing radix tree.

Build a Paged KV Cache Manager From Scratch: A Portfolio Project That Signals Real Systems Skill

Step-by-step build of a real, runnable paged KV cache manager in Python: blocks, block tables, prefix sharing via a radix tree, copy-on-write for beam search, plus a roadmap to senior-level extensions.

Diagram of paged KV cache blocks shared across sequence slots in an LLM inference engine.

Build a Paged KV Cache With Continuous Batching and Prefix Sharing: A From-Scratch Guide

A working engineer’s walkthrough of building a paged KV cache with continuous batching and prefix sharing from scratch — the same ideas vLLM ships in production. Includes runnable Python, an architecture breakdown, tests, and a concrete extension roadmap.

Block diagram of paged KV cache with prefix sharing

Building a Paged KV Cache Manager with Prefix Sharing and Copy-on-Write

Build a from-scratch paged KV cache manager for LLM inference, with block-level prefix sharing and copy-on-write semantics. A real systems project for your CV.

Diagram of vLLM's PagedAttention KV cache allocator mapping logical blocks to physical GPU memory pages.

Optimizing vLLM's PagedAttention Allocator for High-Throughput Batched Inference

A working engineer’s guide to the internals of vLLM’s PagedAttention memory allocator, with concrete knobs, profile traces, and production patterns for squeezing more tokens per second out of a single GPU.