Python

From‑Scratch Safetensors & GGUF Weight Loader with mmap‑Backed Zero‑Copy Views

A practical, runnable guide to constructing a from‑scratch safetensors/GGUF weight loader with zero‑copy mmap views, dtype/quantization decoding, and shape validation, perfect for a CV‑boosting side project.

A diagram of paged attention memory layout

From Scratch: Building a Paged Attention KV Cache for Transformer Inference

A practical build guide for a paged attention KV cache in pure Python, with architecture, code, and extensions for senior-level skills.

A sleek Python code editor with neural network visualisation

Hands‑On MoE Router: Top‑k Gating and Load‑Balancing in Pure Python

A hands‑on guide to implementing a Mixture‑of‑Experts router from scratch in pure Python, with top‑k gating and load‑balancing, ready to run and extend.

A Pure‑Python Continuous‑Batching Inference Engine for LLM Requests

A hands‑on guide to building a pure‑Python continuous‑batching inference engine that packs variable‑length LLM requests into GPU‑friendly batches with adaptive timeout and priority scheduling.

A diagram of a paged attention KV cache manager

Building a PagedAttention KV Cache Manager with Continuous Batching from Scratch

This post walks through building a paged attention KV cache manager with continuous batching, providing runnable Python code and architecture guidance for systems engineers.