Ship a portfolio-grade nanoGPT implementation in pure PyTorch: a tight training loop with AMP, gradient accumulation, cosine LR scheduling, and weight tying — the exact patterns production teams use.
Ship a portfolio project that signals real systems skill: a custom autograd engine, mixed-precision training, gradient checkpointing, and AdamW, all wired into a mini GPT you can actually run.
A runnable, from-scratch implementation of mixed-precision AdamW with dynamic loss scaling and ZeRO-1 sharded optimizer state in pure PyTorch — built to demonstrate real systems engineering, not just model design.
A working engineer’s guide to building a from-scratch mixed-precision training loop — gradient scaling, dynamic loss scaling, and a custom bf16 tensor kernel — applied to a mini GPT. Includes architecture, runnable code, and a roadmap for turning the toy into a portfolio-grade ML systems project.
A working engineer’s guide to building a mini-GPT training loop from scratch in PyTorch — featuring AMP, gradient accumulation, cosine warmup, weight tying, and AdamW — designed as a CV-grade portfolio piece.