PyTorch

A diagram of a neural network merging LoRA weights

From Scratch: Building a LoRA Weight Merger in PyTorch

This tutorial walks you through implementing a LoRA weight merger in PyTorch, giving you a concrete portfolio piece that highlights systems thinking and ML engineering prowess.

A visualization of INT4 quantization layers inside a transformer model, showing weight matrices being compressed and dequantized.

Building a From-Scratch Group-Wise INT4 Quantization Layer for LLaMA Weights

A hands-on guide to implementing group-wise INT4 quantization for LLaMA-style transformers, covering per-group scale calibration, fused dequantization kernels, and benchmarks that demonstrate real ML-systems engineering skill.

A visual representation of memory optimization in neural networks

Building a Gradient Checkpointing Engine from Scratch

A hands-on guide to building a custom gradient checkpointing engine in PyTorch. This side project demonstrates deep memory management and autograd mastery, making your CV stand out to AI infrastructure roles.

GPU architecture diagram showing tiled attention computation with memory hierarchy

Build a FlashAttention-2 Kernel from Scratch in Triton

Build a production-grade FlashAttention-2 kernel from scratch in Triton, complete with tiled causal masking, FP16 accumulation, and PyTorch benchmarks to demonstrate deep GPU programming expertise.

A visualization of ring attention with tokens flowing in a circular topology across GPU devices.

Build a Ring Attention Engine: A Hands-On Guide for Signal-Level Systems Engineers

A hands-on build guide for implementing ring attention — the technique behind near-infinite context transformers — as a portfolio project that signals deep distributed systems and ML engineering skill to hiring managers.