Building Mixed-Precision AdamW with ZeRO-1 from Scratch in Pure PyTorch
A runnable, from-scratch implementation of mixed-precision AdamW with dynamic loss scaling and ZeRO-1 sharded optimizer state in pure PyTorch — built to demonstrate real systems engineering, not just model design.