PyTorch

Illustration of a neural network architecture with attention heads.

Hands‑On Build Guide: Mini‑GPT Training Loop with RoPE, RMSNorm, and GQA

A step‑by‑step guide to training a tiny GPT model from scratch, covering RoPE, RMSNorm, and GQA with runnable PyTorch code.

Build a Minimal GPT Training Loop with Rotary Positional Embeddings

Implement a minimal GPT training loop with rotary positional embeddings from scratch to demonstrate real systems skill to hiring managers.

A visual representation of distributed training sharding optimizer states across multiple GPUs.

Building a Mini GPT with ZeRO Stage 2 from Scratch in PyTorch

Learn how to implement a distributed training loop from scratch, mastering ZeRO Stage 2 and gradient accumulation to optimize memory and compute for large language models.

Diagram of GPU communication rings and tree collectives in NCCL

Inside NCCL Topologies: Collective Algorithms for Multi-GPU Training

Understanding NCCL topologies is essential for optimizing multi-GPU training performance. This post breaks down collective algorithms, hardware-aware patterns, and practical tuning tips.

A developer working on a laptop with code snippets and a model diagram.

Hands‑On LoRA Fine‑Tuning Portfolio Project

Learn how to construct a runnable LoRA fine‑tuning side project that demonstrates production‑grade ML engineering skills and can be highlighted on your CV.