Linformer

Building a Memory-Efficient Attention with Kernel Decomposition: A Hands-On Linformer Project

Implement Linformer attention from scratch in PyTorch, benchmark it, and learn how to extend it for production systems.