Deep Learning

A diagram of a mixture-of-experts router

Build Your Own Mixture-of-Experts Router with Top‑k Gating

This tutorial walks you through building a top‑k gated mixture-of-experts router in PyTorch, giving you a concrete portfolio piece that demonstrates model architecture, sparse routing, and system‑level scaling.

From-Scratch Mini GPT Training Loop with Gradient Checkpointing, Mixed Precision, and a Custom CUDA‑Style Optimizer

A hands‑on guide to training a tiny GPT model end‑to‑end with gradient checkpointing, FP16 mixed precision, and a hand‑rolled optimizer, perfect for a CV‑worthy portfolio.

A sleek neural network diagram overlaid on a laptop screen

Build Your Own Fused Low‑Rank LoRA Fine‑Tuning Engine From Scratch

A hands‑on guide to building a fused low‑rank LoRA fine‑tuning engine from scratch, with runnable PyTorch code, architecture diagrams, and production‑ready extensions.

A neural network diagram showing low-rank weight matrices

From Scratch: Building a LoRA Fine-Tuning Adapter in Pure PyTorch

Implement a Low-Rank Adaptation (LoRA) module from scratch in PyTorch, then extend it with persistence, observability, and distributed training for production readiness.

A block‑sparse attention matrix visualized as a grid of active blocks.

Building a Pure‑Python Block‑Sparse Sliding‑Window Attention (Longformer‑Style)

Implement block‑sparse sliding‑window attention in pure Python to handle long sequences efficiently, mirroring Longformer’s design.