Posts

Stylized diagram of a GPU die with attention tile matrices overlaid in cyan and amber.

Optimizing Triton Kernels for Custom Attention Patterns on H100 GPUs

A working engineer’s guide to squeezing the H100 with custom Triton kernels: tile sizing, swizzle patterns, pipeline stages, and the warp-specialization tricks that make non-vanilla attention fly.

Abstract visualization of attention weights flowing through a transformer.

Language Models Can Now Control Their Own Attention

New techniques let language models decide for themselves where to attend, cutting inference cost without retraining. We break down how gating, sparse kernels, and self-reflective decoding work in production systems.

Diagram of gVisor's sentry and gofer architecture

Implementing gVisor: Inside the User-Space Kernel That Sandboxes Docker Containers

How gVisor reimplements the Linux kernel in user space to safely sandbox containers, and what the trade-offs look like in real deployments.

Diagram of disaggregated LLM serving with a shared KV cache pool across prefill and decode nodes.

Mooncake: Turning KV Cache into a Distributed LLM Serving Layer

Mooncake reframes KV cache as a first-class distributed resource. This post walks through its architecture, the KV store design, and why prefilling and decoding benefit so differently from cache disaggregation.

Abstract visualization of a tensor grid being mapped from float32 values into discrete quantization bins.

Why Quantization Works: The Mathematics That Makes LLMs 4x Smaller

Quantization shrinks 70B-parameter models from 140GB to 35GB without retraining. Here is the math, the failure modes, and why it works in production.