Vllm | martinuke0's Blog

Abstract visualization of distributed GPU clusters processing large language model inference requests

Distributed LLM Inference: Parallelism Strategies

Exploring the core parallelism strategies behind distributed LLM inference — tensor, pipeline, data, and expert parallelism — with real-world architecture patterns and practical deployment insights.

Diagram of paged KV cache blocks shared across sequence slots in an LLM inference engine.

Build a Paged KV Cache With Continuous Batching and Prefix Sharing: A From-Scratch Guide

A working engineer’s walkthrough of building a paged KV cache with continuous batching and prefix sharing from scratch — the same ideas vLLM ships in production. Includes runnable Python, an architecture breakdown, tests, and a concrete extension roadmap.

Diagram of paged-attention KV cache blocks mapped across sequences

Build a Paged-Attention KV Cache Manager: A Hands-On Project That Actually Impresses Hiring Managers

Ship a runnable paged-attention KV cache manager that demonstrates real LLM systems skill: block tables, prefix-sharing across requests, and eviction policies that actually move the needle on memory.

Diagram of vLLM's PagedAttention KV cache allocator mapping logical blocks to physical GPU memory pages.

Optimizing vLLM's PagedAttention Allocator for High-Throughput Batched Inference

A working engineer’s guide to the internals of vLLM’s PagedAttention memory allocator, with concrete knobs, profile traces, and production patterns for squeezing more tokens per second out of a single GPU.

Abstract visualization of a GPU pipeline routing inference requests across multiple LLM instances.

Serving LLMs in Production: A Practical Guide to Latency, Throughput, and Cost

A production-focused walkthrough of how to serve large language models efficiently, covering batching strategies, quantization, KV cache management, and the observability you actually need.