Numpy

Layered neural network diagram showing low-rank adapter matrices branching off frozen weights.

Building a LoRA Training Loop From Scratch in Pure NumPy

Build a working LoRA fine-tuning loop in pure NumPy, computing every gradient by hand through low-rank adapters. A practical side project that demonstrates real systems understanding to hiring managers.

Layered diagram of a quantization pipeline showing calibration, packing, and fused dequantize-matmul.

Build a GPTQ int4 Quantization Engine in Python/NumPy

Ship a runnable int4 weight-only quantization engine from scratch: GPTQ-style calibration, group-wise affine scales, and a fused dequantize-matmul kernel. Built to demonstrate real ML systems skill on a portfolio.

Diagram of a mini GPT training pipeline showing token embeddings, attention, MLP blocks, and an optimizer loop.

Build a Mini GPT From Scratch in Pure NumPy: A Hiring-Manager-Worthy Project

Build a tiny GPT end-to-end in NumPy to demonstrate ownership of the ML stack: mixed precision, AdamW, cosine LR, and gradient accumulation. Code-first, with a roadmap to production-grade upgrades.

Block-tiled attention computation showing memory access patterns across query, key, and value matrices.

Building Flash Attention From Scratch in NumPy: A CV-Grade Side Project

A complete, runnable walkthrough of implementing flash attention from scratch in NumPy, including online softmax, block tiling, causal masking, and correctness checks — designed as a CV-worthy systems project.

Diagram of tiled attention matrix blocks flowing through online softmax accumulation.

Building a FlashAttention-2 Forward Kernel in Pure NumPy

TL;DR — FlashAttention-2 is an IO-aware attention algorithm that fuses the softmax and matrix multiply into tiled CUDA kernels. We rebuild the forward pass from scratch in pure NumPy: blocking the QKV matrices into tiles, running a streaming softmax per block, and rescaling outputs online. The result is a runnable, testable side project that demonstrates memory-hierarchy thinking, numerical stability, and GPU-style programming patterns — exactly what hiring managers look for on a CV. ...