A working engineer’s guide to implementing a speculative decoder from scratch: draft with n-gram proposals, verify with a target model, ship it as a portfolio piece. Real code, real benchmarks, real extensions.
Build a real GGUF parser and tensor loader from scratch in Rust — metadata parsing, q4_0/q8_0 dequantization, mmap’d weight streaming, and a minimal Llama forward pass. The kind of CV project that gets interviews at systems and ML infra teams.
Parse GGUF, dequantize Q4_0 and Q8_0 tensors, and load Llama weights into numpy. A hands-on guide to a side project that demonstrates real systems engineering on a CV.
Build flash attention from scratch in pure NumPy using tiling and the online softmax trick, then extend it toward production. A deep, runnable project that signals real systems engineering to hiring managers.
Explore how tri‑color marking powers low‑pause concurrent collectors, see concrete architecture diagrams, and learn production‑grade heap management tricks.