Mastering vLLM's PagedAttention: Continuous Batching for High-Throughput LLM Inference
vLLM’s PagedAttention treats KV cache like virtual memory pages, eliminating fragmentation and enabling continuous batching that dramatically improves LLM inference throughput.
Build a Pure‑Python Mixture‑of‑Experts Router with Dynamic Load‑Aware Routing and Soft Capacity Sharing
Implement a dynamic MoE router in pure Python, demonstrating load‑balancing, soft capacity sharing, and production‑ready patterns for a standout CV project.
Build a From‑Scratch FlashAttention in Pure NumPy
Implement block‑tiled Q/K/V matmuls with online softmax in pure NumPy, matching naive attention while using O(N) memory; benchmark speed and memory.
Build a Switch Transformer Router with Top‑k Gating and Auxiliary Load‑Balancing in Pure Python
Learn how to implement a switch transformer router with top‑k gating, expert capacity, and auxiliary load‑balancing loss in pure Python, and why this project signals real systems skill to hiring managers.
Dynamic Prompt‑Window Manager with Sliding‑Window Context Caching
Learn how to construct a sliding‑window context cache with importance‑based eviction, a portable project that showcases systems engineering skill for LLM‑focused roles.