Systems

A diagram of a speculative decoding pipeline

Speculative Decoding Engine with Draft-Model Verification and KV-Cache Sharing in Pure Python

This guide walks through building a speculative decoding engine in pure Python, including draft-model verification and KV-cache sharing. It’s ideal for showcasing systems skills to hiring managers.

A stylized graph of nodes and edges representing an HNSW index

Building a HNSW Vector Index from Scratch in Python

Learn to build a HNSW‑based vector index from scratch, demonstrating systems skills that catch hiring managers’ eyes.

Diagram of a paged KV cache with request preemption

Building a Continuous Batching LLM Serving Engine with Paged KV Cache and Request Preemption

Learn how to design and implement a high‑throughput LLM serving backend that batches requests continuously, uses a paged KV cache for memory efficiency, and preempts low‑priority requests to maximize throughput.

Cover image depicting a vector index architecture

Building a Pure-Python IVF-PQ Vector Index with On-Disk Inverted Files

A hands-on guide to implementing an IVF-PQ vector index in pure Python, with on-disk inverted files and product quantization for scalable similarity search.

A diagram of a transformer model with highlighted KV cache

Building a Pure‑Python Int4 KV‑Cache for Transformer Inference

This project demonstrates low‑level memory optimization, quantization, and efficient inference techniques that signal strong systems engineering expertise to hiring managers.