High-Performance Computing

Deep Dive into NVIDIA NVLink: Designing High-Bandwidth GPU Topologies for Clusters

A technical guide to NVLink and NVSwitch, covering topology design, bandwidth scaling, and production patterns for GPU clusters.

Python code and matrix multiplication diagrams on a dark terminal screen.

Build a FlashAttention-2 Inference Kernel from Scratch in Pure Python

Learn how to build a FlashAttention-2 inference kernel from scratch in pure Python. This guide covers tiled block-sparse causal attention and fused softmax to signal deep systems engineering skills.

NVIDIA Hopper GPU architecture with sparse tensor cores and memory hierarchy illustration

Architecting Sparse Tensor Computations on NVIDIA Hopper: A Guide to Efficient Memory Management

A practical guide to managing memory and optimizing sparse tensor operations on NVIDIA Hopper GPUs for AI and HPC workloads.

Short description of the cover image subject.

Building a From-Scratch INT8 Strided Convolution Kernel

Learn how to build a high-performance INT8 convolution kernel from scratch using strided memory access and SIMD instructions.

CUDA Graph workflow illustrating node dependencies and kernel launch streams

Mastering CUDA Graphs: Low-Overhead Kernel Launching for High-Performance Computing

Learn how CUDA Graphs reduce kernel launch overhead, enable dynamic task scheduling, and improve throughput in high-performance computing applications.