Flash-Attention

Diagram of tiled self-attention computation

Building a FlashAttention-Style Tiled Self-Attention Block in Pure NumPy

Implement a FlashAttention‑style tiled self‑attention block with online softmax in pure NumPy, demonstrating systems engineering proficiency.

Diagram of grouped-query attention with rotary position embeddings feeding into a tiled Flash-Attention forward pass.

Building Grouped-Query Attention with RoPE and Flash-Attention from Scratch

Step-by-step tutorial for implementing GQA, RoPE, and an online-softmax Flash-Attention forward pass in pure PyTorch — a portfolio project that signals real systems-level ML skill.

Block-tiled attention computation showing memory access patterns across query, key, and value matrices.

Building Flash Attention From Scratch in NumPy: A CV-Grade Side Project

A complete, runnable walkthrough of implementing flash attention from scratch in NumPy, including online softmax, block tiling, causal masking, and correctness checks — designed as a CV-worthy systems project.

Code editor showing a Python implementation of flash attention with tiling and online softmax in NumPy.

Building Flash Attention from Scratch in Pure NumPy: A CV-Worthy Systems Project

Build flash attention from scratch in pure NumPy using tiling and the online softmax trick, then extend it toward production. A deep, runnable project that signals real systems engineering to hiring managers.

Tiled matrix multiplication diagram with softmax rows

Building Flash Attention from Scratch in NumPy: A CV-Worthy Side Project

A working engineer’s guide to implementing Flash Attention from the ground up using tiling and the online softmax trick — runnable NumPy, end-to-end tests, and a roadmap to senior-level extensions.