Transformer

A BM25 Sparse Retriever + Tiny Transformer Generator RAG Pipeline in Pure Python

This guide walks you through implementing a BM25 sparse retriever and a tiny decoder‑only transformer generator from scratch in pure Python, producing a functional RAG pipeline you can showcase on your CV.

Build a Minimal Transformer Inference Engine for Your Portfolio

A practical guide to building a minimal transformer inference engine with KV cache, tokenizer, attention, and sampler. It demonstrates systems skills that hiring managers look for.

A diagram of a transformer with LoRA adapters

From‑Scratch LoRA Adapter Injection and Weight Merging in Pure Python

Implement LoRA adapter injection and weight merging from scratch in pure Python, demonstrating transformer systems engineering for hiring managers.

A visual representation of distributed training sharding optimizer states across multiple GPUs.

Building a Mini GPT with ZeRO Stage 2 from Scratch in PyTorch

Learn how to implement a distributed training loop from scratch, mastering ZeRO Stage 2 and gradient accumulation to optimize memory and compute for large language models.

A diagram of a transformer with paged attention and LRU cache.

Building a Pure-Python Paged-Attention Transformer with LRU KV-Cache Eviction

Implement a paged-attention transformer with LRU KV-cache eviction in pure Python, demonstrating systems engineering proficiency.