Gpt | martinuke0's Blog

Illustration of a neural network with checkpoint arrows

Hands‑On Activation Checkpointing for GPT Training From Scratch

Implement activation checkpointing for GPT training from scratch, with runnable Python code and production‑grade insights.

Illustration of a neural network architecture with attention heads.

Hands‑On Build Guide: Mini‑GPT Training Loop with RoPE, RMSNorm, and GQA

A step‑by‑step guide to training a tiny GPT model from scratch, covering RoPE, RMSNorm, and GQA with runnable PyTorch code.

Rotary embedding circle over a GPT schematic

Hands‑On Build Guide: Rotary Positional Embedding for a Tiny GPT

A step‑by‑step implementation of a tiny GPT with rotary positional embeddings, from architecture to runnable code, perfect for demonstrating systems skill to hiring managers.

Diagram of a mini GPT training pipeline showing token embeddings, attention, MLP blocks, and an optimizer loop.

Build a Mini GPT From Scratch in Pure NumPy: A Hiring-Manager-Worthy Project

Build a tiny GPT end-to-end in NumPy to demonstrate ownership of the ML stack: mixed precision, AdamW, cosine LR, and gradient accumulation. Code-first, with a roadmap to production-grade upgrades.