Triton

Deep Dive into NVIDIA Triton Inference Server: Scaling Multi‑Model GPU Workloads

NVIDIA Triton Inference Server is a critical tool for scaling AI in production. This deep dive explores its architecture, multi-model concurrency, and deployment patterns.

GPU architecture diagram showing tiled attention computation with memory hierarchy

Build a FlashAttention-2 Kernel from Scratch in Triton

Build a production-grade FlashAttention-2 kernel from scratch in Triton, complete with tiled causal masking, FP16 accumulation, and PyTorch benchmarks to demonstrate deep GPU programming expertise.

Short description of the cover image subject.

Build a FlashAttention‑2 Engine from Scratch in Triton

A hands‑on guide to implementing FlashAttention‑2 in Triton, complete with tiled kernels, causal masking, and PyTorch autograd wiring — perfect for a standout CV project.

Stylized diagram of a GPU die with attention tile matrices overlaid in cyan and amber.

Optimizing Triton Kernels for Custom Attention Patterns on H100 GPUs

A working engineer’s guide to squeezing the H100 with custom Triton kernels: tile sizing, swizzle patterns, pipeline stages, and the warp-specialization tricks that make non-vanilla attention fly.