Posts

A heatmap-style illustration of GPU SM utilization across batched transformer layers.

Scaling TensorRT-LLM: Kernel Fusion and Inflight Batching in Production

How NVIDIA’s TensorRT-LLM squeezes maximum throughput from a single GPU — and across a fleet — by fusing kernels, paged KV-cache, and continuously batching requests.

Diagram showing QUIC packet flow with ACK frames and congestion window feedback loop.

Inside QUIC's Loss Recovery and Congestion Control Machinery in Production Deployures

A deep dive into the loss detection, ACK handling, and congestion control loops inside QUIC, with a focus on how they behave under real production traffic on HTTP/3 and gRPC.

Diagram of vLLM's PagedAttention KV cache allocator mapping logical blocks to physical GPU memory pages.

Optimizing vLLM's PagedAttention Allocator for High-Throughput Batched Inference

A working engineer’s guide to the internals of vLLM’s PagedAttention memory allocator, with concrete knobs, profile traces, and production patterns for squeezing more tokens per second out of a single GPU.

Stylized diagram of a columnar vector being processed by SIMD lanes.

Inside DuckDB's Vectorized Execution Engine: How Columnar Storage Meets SIMD-Aware Query Processing

How DuckDB combines a columnar in-memory layout with vectorized, SIMD-aware operators to deliver analytical query performance that rivals dedicated warehouses — and why the same ideas are reshaping mainstream engines like Velox and DataFusion.

Diagram of a continuous batching scheduler with paged KV cache blocks being shared across active sequences.

Build a Continuous Batching LLM Scheduler From Scratch

Ship a working continuous batching scheduler with paged KV cache and dynamic request interleaving. Real Python, runnable locally, and a roadmap from toy to production-flavored.