Optimization

From-Scratch Mini GPT Training Loop with Gradient Checkpointing, Mixed Precision, and a Custom CUDA‑Style Optimizer

A hands‑on guide to training a tiny GPT model end‑to‑end with gradient checkpointing, FP16 mixed precision, and a hand‑rolled optimizer, perfect for a CV‑worthy portfolio.

Inside LLVM's New Pass Manager: A Deep Dive into Its Architecture

Exploring LLVM’s new pass manager: architecture, design patterns, and how it transforms modern compiler optimization.

Short description of the cover image subject.

Deep Dive into CUDA Graphs for Efficient Kernel Launch Overhead Reduction

CUDA graphs reduce kernel launch overhead by batching operations. They can deliver up to 40% speedups in real-world inference pipelines.

Illustration of a CPU core with SIMD lanes processing data.

Why Your Compiler Cannot Vectorize That Loop

A deep dive into the reasons behind failed auto‑vectorization and actionable steps to write loops the compiler can turn into SIMD.

How Modern Compilers Eliminate Branch Mispredictions Using Predication

Explore how compilers replace hard-to‑predict branches with predicated instructions, the underlying hardware mechanisms, and real‑world performance results.