Autograd

Illustration of a neural network training loop

Mini GPT Training Loop with Custom Autograd: A Hands-On Portfolio Project

A step‑by‑step guide to implement a tiny GPT training loop with custom autograd, perfect for a CV side project that showcases low‑level ML engineering skills.

Diagram of a mixed-precision training loop with gradient checkpointing for a mini GPT.

Build a Mini GPT From Scratch: A Mixed-Precision Training Loop That Actually Impresses

Ship a portfolio project that signals real systems skill: a custom autograd engine, mixed-precision training, gradient checkpointing, and AdamW, all wired into a mini GPT you can actually run.

Feedback