Llm | martinuke0's Blog

Building a Paged KV Cache Allocator with Continuous Batching in Pure Python

This project demonstrates memory management, concurrent request scheduling, and low-level allocator design—all core skills for backend and ML infrastructure roles.

A diagram of a transformer with paged attention and LRU cache.

Building a Pure-Python Paged-Attention Transformer with LRU KV-Cache Eviction

Implement a paged-attention transformer with LRU KV-cache eviction in pure Python, demonstrating systems engineering proficiency.

A visualization of a KV cache data structure with key and value tensors being incrementally appended during autoregressive LLM decoding.

Building a Pure Python Incremental KV Cache for Efficient Long-Context LLM Inference

Build a production-grade incremental KV cache from scratch in pure Python. This guide covers architecture, implementation, testing, and a roadmap to production-flavored features that signal real systems skill to hiring managers.

A code editor showing a Python sampler class with visualization of probability distributions

Build an LLM Decoding Sampler from Scratch: A Portfolio Project That Signals Real Systems Skill

A hands-on guide to building a from-scratch LLM decoding sampler implementing temperature, top-k, top-p, and contrastive search in pure Python. Includes architecture, complete code, and a roadmap to senior-level extensions.

A conceptual illustration of speculative decoding with parallel paths.

Building a Pure-Python Speculative Decoding Engine for Your Portfolio

Build a pure‑Python speculative decoding engine that uses a draft LLM and parallel acceptance‑reject sampling to speed up text generation. This project highlights real systems engineering skills for your CV.