Kv-Cache

Build a Paged‑Attention KV Cache with Block‑Level LFU Eviction in Python

Implement a production‑inspired KV cache with paged attention and LFU block eviction; source code included for portfolio demonstration.

Python code on a dark terminal, visualizing paged KV cache buckets

Build a Paged Attention KV Cache with H2O Eviction in Pure Python

Learn how to implement a paged attention KV cache with H2O eviction in pure Python, a concrete project that demonstrates low‑latency data structures, eviction policies, and production‑ready patterns for AI/ML infrastructure.

Memory‑Aware KV Cache Eviction Engine for Long‑Context LLM Inference

A hands‑on guide to constructing a pure‑Python KV cache eviction engine that uses attention‑based pruning to reduce memory footprint for long‑context LLM inference.

A diagram of paged attention memory layout

From Scratch: Building a Paged Attention KV Cache for Transformer Inference

A practical build guide for a paged attention KV cache in pure Python, with architecture, code, and extensions for senior-level skills.

A diagram of a paged attention KV cache manager

Building a PagedAttention KV Cache Manager with Continuous Batching from Scratch

This post walks through building a paged attention KV cache manager with continuous batching, providing runnable Python code and architecture guidance for systems engineers.