PyTorch

Neural network diagram with attention heads visualized as colored connections between token nodes.

Building LLMs from Scratch: A Working Engineer's Guide

A working engineer’s end-to-end guide to assembling a transformer language model in PyTorch: from BPE tokenization to multi-head attention, rotary embeddings, and a real training loop on a single GPU.

Diagram of a speculative decoder with Medusa draft heads feeding a verification pass on a target model.

Building a Speculative Decoder with Medusa-Style Draft Heads in PyTorch

Build a speculative decoder with Medusa-style draft heads and modern samplers in PyTorch. A production-flavored portfolio project that signals real ML systems skill.

Stylized diagram of a rotary positional embedding applied to a query and key tensor on a GPU

Building a Rotary Positional Embedding Kernel from Scratch: A Portfolio Project That Signals Real Systems Skill

Build a real rotary positional embedding kernel in PyTorch with a custom autograd node, fused CUDA-style kernels, and a tiny test harness. The project you ship on a CV to signal GPU, systems, and ML-engineering depth.

Build a Token Sampler From Scratch: Temperature, Top-k, Top-p, Min-p, and Contrastive Search

Build a streaming token sampler from scratch in PyTorch covering temperature, top-k, top-p, min-p, and contrastive search — a CV-worthy systems project for ML engineers.

Diagram of mixed-precision AdamW with ZeRO-1 sharding across GPUs.

Building Mixed-Precision AdamW with ZeRO-1 from Scratch in Pure PyTorch

A runnable, from-scratch implementation of mixed-precision AdamW with dynamic loss scaling and ZeRO-1 sharded optimizer state in pure PyTorch — built to demonstrate real systems engineering, not just model design.