Attention

Hands-on Build Guide: Tiled Self-Attention Kernel with Online Softmax in Pure NumPy

A step‑by‑step NumPy implementation of tiled soft‑max attention, mirroring FlashAttention’s efficiency, perfect for a CV‑boosting side project.

A sleek Python code editor with visual cache diagrams

Pure Python Paged Attention Engine with LRU KV‑Cache Eviction

A step‑by‑step guide to building a pure Python paged attention engine with LRU cache eviction, perfect for demonstrating systems‑level skills to ML recruiters.

Python code on a dark terminal, visualizing paged KV cache buckets

Build a Paged Attention KV Cache with H2O Eviction in Pure Python

Learn how to implement a paged attention KV cache with H2O eviction in pure Python, a concrete project that demonstrates low‑latency data structures, eviction policies, and production‑ready patterns for AI/ML infrastructure.

Build an Attention Block from Scratch: A Portfolio Project for Systems Engineers

Learn to implement a transformer attention block from scratch in Python, package it as a service, and extend it with production-grade features.

A block‑sparse attention matrix visualized as a grid of active blocks.

Building a Pure‑Python Block‑Sparse Sliding‑Window Attention (Longformer‑Style)

Implement block‑sparse sliding‑window attention in pure Python to handle long sequences efficiently, mirroring Longformer’s design.