Implementing CUDA Graphs for Low-Latency GPU Inference Pipelines
CUDA Graphs capture and replay GPU workloads, reducing kernel launch overhead and cutting inference latency by up to 40% in production systems.
Gpu
Deep Dive into CUDA Graphs for Efficient Kernel Launch Overhead Reduction
CUDA graphs reduce kernel launch overhead by batching operations. They can deliver up to 40% speedups in real-world inference pipelines.
Build a FlashAttention‑2 Engine from Scratch in Triton
A hands‑on guide to implementing FlashAttention‑2 in Triton, complete with tiled kernels, causal masking, and PyTorch autograd wiring — perfect for a standout CV project.
Optimizing GPU Memory Pools: Inside PyTorch's CUDA Caching Allocator
PyTorch’s CUDA caching allocator hides the cost of repeated allocations by recycling VRAM in fixed-size blocks. Here’s how it works, why it matters at scale, and how to squeeze more performance out of it.
GPU Architecture for Backend Engineers: Why Your Workloads Run on Massively Parallel Silicon
A practical tour of GPU architecture for engineers who don’t write shaders: how streaming multiprocessors, warps, and the memory hierarchy actually work, and why that matters for inference, analytics, and simulation workloads.