A sleek notebook with code and diagrams

From‑Scratch MoE Transformer Block with Sinkhorn Routing in NumPy

A hands‑on guide to building a Mixture‑of‑Experts transformer block from scratch in NumPy. It features Sinkhorn‑based routing and balanced expert load, ready to showcase on a CV.

November 22, 2026 · 8 min · 1554 words · martinuke0
Feedback