Deep Learning

Illustration of a neural network architecture with attention heads.

Hands‑On Build Guide: Mini‑GPT Training Loop with RoPE, RMSNorm, and GQA

A step‑by‑step guide to training a tiny GPT model from scratch, covering RoPE, RMSNorm, and GQA with runnable PyTorch code.

GPTQ quantization engine pipeline diagram

Building a GPTQ Weight Quantization Engine from Scratch: Fisher Scoring & Mixed-Precision Calibration

Learn to build a functional GPTQ quantization engine from scratch, complete with Fisher information scoring and mixed-precision calibration, and how to signal systems-level engineering skill to hiring managers.

Build a Minimal GPT Training Loop with Rotary Positional Embeddings

Implement a minimal GPT training loop with rotary positional embeddings from scratch to demonstrate real systems skill to hiring managers.

A visual representation of distributed training sharding optimizer states across multiple GPUs.

Building a Mini GPT with ZeRO Stage 2 from Scratch in PyTorch

Learn how to implement a distributed training loop from scratch, mastering ZeRO Stage 2 and gradient accumulation to optimize memory and compute for large language models.

A conceptual diagram of speculative decoding showing a fast draft model feeding tokens to a larger verifier model.

Build a Speculative Decoding Engine: A Portfolio Project That Signals Real Systems Skill

A hands-on guide to building a speculative decoding engine with a draft model and verifier in pure Python. This project signals deep systems expertise and is practical enough to actually complete, with real runnable code at every step.