Build a tiny GPT end-to-end in NumPy to demonstrate ownership of the ML stack: mixed precision, AdamW, cosine LR, and gradient accumulation. Code-first, with a roadmap to production-grade upgrades.
How the Linux kernel’s KFIFO achieves a wait-free, lock-free FIFO with a single atomic variable. We walk through the ringbuffer math, the memory-ordering subtlety that makes it correct on weakly ordered CPUs, and what you can borrow for your own high-throughput pipelines.
A complete, runnable walkthrough of implementing flash attention from scratch in NumPy, including online softmax, block tiling, causal masking, and correctness checks — designed as a CV-worthy systems project.
A deep dive into the architecture behind JetStream’s exactly-once semantics — covering Raft-replicated streams, per-consumer ack tracking, redelivery semantics, and the dedup window that ties it all together.
A working engineer’s guide to squeezing the H100 with custom Triton kernels: tile sizing, swizzle patterns, pipeline stages, and the warp-specialization tricks that make non-vanilla attention fly.