Python

Cover image showing BPE tokenization process

Building a BPE Tokenizer from Scratch in Python

Implement a byte-level BPE tokenizer from scratch, test it, and extend it to production-grade tooling.

A visualization of a sliding window compressing a long prompt into a fixed-length context window.

Build a Sliding-Window Prompt Compressor from Scratch in Pure Python

A hands-on guide to building a sliding-window prompt compressor from scratch in pure Python, using a learned importance scorer to evict low-relevance tokens and keep attention focused within a fixed context budget.

A diagram of a RAG pipeline with BM25 and transformer

A BM25 Sparse Retriever + Tiny Transformer Generator RAG Pipeline in Pure Python

This guide walks you through implementing a BM25 sparse retriever and a tiny decoder‑only transformer generator from scratch in pure Python, producing a functional RAG pipeline you can showcase on your CV.

A visualization of LLM sampling distributions at different temperature settings, showing probability mass shifting from peaked to flat.

Build a From-Scratch LLM Sampler: Temperature, Top-k/Top-p, and Multinomial Sampling vs. Hugging Face

A hands-on guide to building a from-scratch LLM sampler with temperature scaling, top-k/top-p warping, and multinomial sampling, benchmarked against Hugging Face. Includes runnable code and a roadmap to production.

Build a GGUF Parser with Memory-Mapped Quantized Tensor Loading in Pure Python

Build a production-grade GGUF parser from scratch in pure Python, featuring memory-mapped quantized tensor loading. Learn the format internals, write real dequantization kernels, and discover why this project stands out to hiring managers.