Kv-Cache

Building a Sliding-Window Attention KV Cache with Dynamic Token Pruning

This guide walks through implementing a sliding-window attention KV cache with dynamic token pruning, providing runnable Python code and tests.

A diagram of a transformer with paged attention and LRU cache.

Building a Pure-Python Paged-Attention Transformer with LRU KV-Cache Eviction

Implement a paged-attention transformer with LRU KV-cache eviction in pure Python, demonstrating systems engineering proficiency.

A visualization of a KV cache data structure with key and value tensors being incrementally appended during autoregressive LLM decoding.

Building a Pure Python Incremental KV Cache for Efficient Long-Context LLM Inference

Build a production-grade incremental KV cache from scratch in pure Python. This guide covers architecture, implementation, testing, and a roadmap to production-flavored features that signal real systems skill to hiring managers.

A conceptual diagram of a paged attention KV-cache architecture showing memory blocks, page tables, and eviction queues.

Build a Pure-Python Paged Attention KV-Cache with LRU and Frequency-Based Eviction

A hands-on guide to building a pure-Python paged attention KV-cache with LRU and frequency-based eviction policies — a portfolio project that demonstrates deep systems engineering skills.

Python code editor with cache manager diagram

Pure‑Python Paged Attention KV‑Cache Manager: LRU & Random Eviction

Build a lightweight, pure‑Python KV‑cache manager that implements LRU and random eviction strategies, complete with paging, testing, and extension pathways for senior‑level topics.