Learn how to implement a fully differentiable top‑k gating network in NumPy, enabling you to prototype mixture‑of‑experts layers and showcase a concrete, production‑ready side project on your CV.
Implement activation checkpointing for GPT training from scratch, with runnable Python code and production‑grade insights.
Implement Linformer attention from scratch in PyTorch, benchmark it, and learn how to extend it for production systems.
Implement a sliding‑window attention block in PyTorch, test it, and extend it with distributed training, mixed precision, and monitoring to showcase systems expertise.
A hands-on build guide for a LoRA adapter trainer with fused low-rank gradients and adapter-aware inference in pure NumPy — the kind of project that signals deep ML and systems expertise to hiring managers.