Kv-Cache

Streaming KV Cache with Attention Sinks for Infinite Context in Pure Python

Learn to implement a streaming KV cache with attention sinks in Python, enabling infinite context for large language models.

A visual representation of compressed KV cache attention with memory tiers

Build a Memory-Compressed Attention Engine with KV Caching from Scratch

A hands-on build guide for implementing a memory-compressed attention engine with key-value caching from scratch in Python. Includes architecture diagrams, production-grade code, and a roadmap to senior-level extensions.

A diagram of a KV cache with entries being evicted

Build a KV Cache Eviction Engine for LLM Context Windows

Implement a pluggable KV cache eviction engine from scratch, demonstrating systems thinking and production-ready patterns.

A code editor displaying Python implementation of paged attention and KV-cache management

Build a Minimal LLM Inference Engine with Paged Attention and Adaptive KV-Cache Eviction in Pure Python

A hands-on guide to building a minimal LLM inference engine with paged attention and adaptive KV-cache eviction in pure Python, designed to demonstrate real systems engineering prowess on your CV.

A diagram of a transformer model with highlighted KV cache

Building a Pure‑Python Int4 KV‑Cache for Transformer Inference

This project demonstrates low‑level memory optimization, quantization, and efficient inference techniques that signal strong systems engineering expertise to hiring managers.