Hands-on Build Guide: Tiled Self-Attention Kernel with Online Softmax in Pure NumPy
A step‑by‑step NumPy implementation of tiled soft‑max attention, mirroring FlashAttention’s efficiency, perfect for a CV‑boosting side project.
Numpy
From‑Scratch Speculative Decoding Engine in Pure Python
A hands‑on guide to building a from‑scratch speculative decoding engine in pure Python, with draft‑then‑verify sampling, temperature, top‑k and nucleus (top‑p) sampling, all using only numpy. Perfect for a CV‑boosting side project.
From-Scratch Mini GPT Training Loop with Gradient Checkpointing, Mixed Precision, and a Custom CUDA‑Style Optimizer
A hands‑on guide to training a tiny GPT model end‑to‑end with gradient checkpointing, FP16 mixed precision, and a hand‑rolled optimizer, perfect for a CV‑worthy portfolio.
Build a Differentiable Top‑k Gating Network for MoE Layers from Scratch with NumPy
Learn how to implement a fully differentiable top‑k gating network in NumPy, enabling you to prototype mixture‑of‑experts layers and showcase a concrete, production‑ready side project on your CV.
From‑Scratch Differentiable Sparse Mixture‑of‑Experts Gating Layer with Load Balancing in NumPy
A step‑by‑step guide to implementing a sparse Mixture‑of‑Experts gating layer in NumPy, demonstrating differentiable programming, load‑balancing, and production‑ready patterns for engineering roles.