Transformers

A sleek notebook with code snippets and a transformer diagram.

Hands-On Build Guide: Minimal LoRA Fine‑Tuning Engine for a 2‑Layer Transformer

A practical, end‑to‑end guide to building a minimal LoRA fine‑tuning engine for a 2‑layer transformer, with runnable code and production‑ready extensions.

A diagram of paged attention memory layout

From Scratch: Building a Paged Attention KV Cache for Transformer Inference

A practical build guide for a paged attention KV cache in pure Python, with architecture, code, and extensions for senior-level skills.

A visual representation of compressed KV cache attention with memory tiers

Build a Memory-Compressed Attention Engine with KV Caching from Scratch

A hands-on build guide for implementing a memory-compressed attention engine with key-value caching from scratch in Python. Includes architecture diagrams, production-grade code, and a roadmap to senior-level extensions.

A code editor displaying a Python sampling loop with temperature and top-k parameters

Build a From-Scratch LLM Sampler: Temperature, Top-k, Top-p in Pure Python

A hands-on guide to building a pure-Python transformer model with a custom decoding loop implementing temperature, top-k, and top-p sampling — a CV-worthy project that demonstrates deep ML systems knowledge.

A diagram of a transformer model with highlighted KV cache

Building a Pure‑Python Int4 KV‑Cache for Transformer Inference

This project demonstrates low‑level memory optimization, quantization, and efficient inference techniques that signal strong systems engineering expertise to hiring managers.