Eviction

Memory‑Aware KV Cache Eviction Engine for Long‑Context LLM Inference

A hands‑on guide to constructing a pure‑Python KV cache eviction engine that uses attention‑based pruning to reduce memory footprint for long‑context LLM inference.

Python code editor with cache manager diagram

Pure‑Python Paged Attention KV‑Cache Manager: LRU & Random Eviction

Build a lightweight, pure‑Python KV‑cache manager that implements LRU and random eviction strategies, complete with paging, testing, and extension pathways for senior‑level topics.

Redis Cluster architecture diagram showing memory distribution across shards

Optimizing Redis Cluster Eviction: Predictable Latency Under Memory Pressure

Redis Cluster eviction under memory pressure is a common source of unpredictable latency spikes. This post breaks down the eviction mechanics, identifies the failure modes, and provides concrete strategies to keep p99 latency stable.