Building a Memory-Efficient Attention with Kernel Decomposition: A Hands-On Linformer Project
Implement Linformer attention from scratch in PyTorch, benchmark it, and learn how to extend it for production systems.
Attention
Streaming KV Cache with Attention Sinks for Infinite Context in Pure Python
Learn to implement a streaming KV cache with attention sinks in Python, enabling infinite context for large language models.
Building a Sliding Window Attention Block from Scratch: A Portfolio Project for Systems Engineers
Implement a sliding‑window attention block in PyTorch, test it, and extend it with distributed training, mixed precision, and monitoring to showcase systems expertise.
Hands-On Build Guide: Flash Attention Implementation from Scratch in NumPy
A step-by-step NumPy guide to implementing Flash Attention with tiled multi-head self-attention and causal masking, perfect for a portfolio CV.
Build a Minimal Transformer Inference Engine for Your Portfolio
A practical guide to building a minimal transformer inference engine with KV cache, tokenizer, attention, and sampler. It demonstrates systems skills that hiring managers look for.