Performance

A diagram of a speculative decoding pipeline

Speculative Decoding Engine with Draft-Model Verification and KV-Cache Sharing in Pure Python

This guide walks through building a speculative decoding engine in pure Python, including draft-model verification and KV-cache sharing. It’s ideal for showcasing systems skills to hiring managers.

Diagram of a paged KV cache with request preemption

Building a Continuous Batching LLM Serving Engine with Paged KV Cache and Request Preemption

Learn how to design and implement a high‑throughput LLM serving backend that batches requests continuously, uses a paged KV cache for memory efficiency, and preempts low‑priority requests to maximize throughput.

Short description of the cover image subject.

Implementing CUDA Graphs for Low-Latency GPU Inference Pipelines

CUDA Graphs capture and replay GPU workloads, reducing kernel launch overhead and cutting inference latency by up to 40% in production systems.

Redis cache stampede visualization showing concurrent requests collapsing into a single backend call

Optimizing Redis Cache Stampede Protection: Request Coalescing and Probabilistic Early Expiration

Learn how request coalescing and probabilistic early expiration work together to eliminate cache stampede in Redis-backed systems, with production-grade implementation patterns and real failure scenarios.

Build a BPE Tokenizer from Scratch: Parallel Vocab Builder, Priority‑Heap Merge Scheduler, Subword Regularization, and Streaming Encoder with LRU Caching

A hands‑on guide to building a production‑grade BPE tokenizer that demonstrates parallel processing, algorithmic scheduling, and caching strategies for hiring managers.