Continuous-Batching

A sleek terminal showing a Python script scheduling LLM requests

Continuous Batching LLM Scheduler: Hands‑On Build Guide

A step‑by‑step guide to building a runnable continuous batching LLM scheduler that you can showcase on your CV, with architecture, code, testing, and extension ideas.

A diagram of a paged attention KV cache manager

Building a PagedAttention KV Cache Manager with Continuous Batching from Scratch

This post walks through building a paged attention KV cache manager with continuous batching, providing runnable Python code and architecture guidance for systems engineers.

Building a Paged KV Cache Allocator with Continuous Batching in Pure Python

This project demonstrates memory management, concurrent request scheduling, and low-level allocator design—all core skills for backend and ML infrastructure roles.

Code editor showing a Python-based LLM serving engine implementation

Build a Pure-Python Continuous Batching LLM Serving Engine from Scratch

A hands-on guide to building a pure-Python continuous batching LLM serving engine with PagedAttention KV cache and dynamic batch scheduling — a portfolio project that signals deep systems engineering skill.

Diagram of a continuous batching scheduler with paged KV cache blocks being shared across active sequences.

Build a Continuous Batching LLM Scheduler From Scratch

Ship a working continuous batching scheduler with paged KV cache and dynamic request interleaving. Real Python, runnable locally, and a roadmap from toy to production-flavored.