https://martinuke0.github.io/tags/switch-transformer/ Recent content in Switch-Transformer on martinuke0's Blog https://martinuke0.github.io/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E https://martinuke0.github.io/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E Hugo -- 0.152.2 en Thu, 10 Sep 2026 07:01:49 +0000 https://martinuke0.github.io/posts/2026-09-10-switch-transformer-style-sparse-moe-layer-from-scratch-in-pytorch/ Thu, 10 Sep 2026 07:01:49 +0000 https://martinuke0.github.io/posts/2026-09-10-switch-transformer-style-sparse-moe-layer-from-scratch-in-pytorch/ Build a Switch Transformer sparse MoE layer in PyTorch with differentiable routing, capacity limits, and load-balancing loss for a portfolio project.