Rotary-Positional-Embeddings

Rotary embedding circle over a GPT schematic

Hands‑On Build Guide: Rotary Positional Embedding for a Tiny GPT

A step‑by‑step implementation of a tiny GPT with rotary positional embeddings, from architecture to runnable code, perfect for demonstrating systems skill to hiring managers.

Stylized diagram of a rotary positional embedding applied to a query and key tensor on a GPU

Building a Rotary Positional Embedding Kernel from Scratch: A Portfolio Project That Signals Real Systems Skill

Build a real rotary positional embedding kernel in PyTorch with a custom autograd node, fused CUDA-style kernels, and a tiny test harness. The project you ship on a CV to signal GPU, systems, and ML-engineering depth.

Feedback