A hands-on guide to implementing a sparse MoE router from scratch in NumPy, covering noisy top-k gating, capacity-factor batching, and auxiliary load-balancing loss — a CV-worthy side project for ML systems engineers.
Build a sparse Mixture‑of‑Experts inference engine from scratch in Python, mastering top‑k gating, capacity‑constrained dispatch, and load‑balancing loss.
A hands‑on guide to building a LoRA adapter layer in pure PyTorch, with complete code, merge/unmerge, and tips for turning the project into a CV‑standing side project.
A hands-on guide to building a multi-head self-attention block with RoPE and ALiBi from scratch, signaling advanced ML and systems engineering skills to hiring managers.
Implement a Switch Transformer‑style sparse mixture‑of‑experts layer in PyTorch, complete with differentiable routing, capacity limits, and load‑balancing loss.