Inference

A schematic of multiple LoRA adapters connected to a shared base model.

Multi-Adapter LoRA Inference Engine with Weight Paging and Shared Base

A hands‑on guide to building a multi‑adapter LoRA inference engine with weight paging and a shared base model, perfect for showcasing systems engineering skills.

A diagram of paged attention memory layout

From Scratch: Building a Paged Attention KV Cache for Transformer Inference

A practical build guide for a paged attention KV cache in pure Python, with architecture, code, and extensions for senior-level skills.

A Pure‑Python Continuous‑Batching Inference Engine for LLM Requests

A hands‑on guide to building a pure‑Python continuous‑batching inference engine that packs variable‑length LLM requests into GPU‑friendly batches with adaptive timeout and priority scheduling.

A diagram of a paged attention KV cache manager

Building a PagedAttention KV Cache Manager with Continuous Batching from Scratch

This post walks through building a paged attention KV cache manager with continuous batching, providing runnable Python code and architecture guidance for systems engineers.

Python code on a laptop screen with neural network diagrams

Hands‑On Build Guide: Speculative Decoding Inference Engine in Python

A step‑by‑step tutorial to build a speculative decoding inference engine in pure Python, demonstrating model parallelism, benchmarking, and production‑grade extensions.