Llm | martinuke0's Blog

Abstract visualization of token sampling probabilities

Build a Pure Python LLM Decoder from Scratch: Temperature, Top-k, and Top-p Sampling

A hands-on guide to implementing temperature, top-k, and top-p sampling in pure Python, with runnable code that showcases systems engineering skills.

Abstract illustration of a neural network with retrieval arrows

Building a Speculative RAG Decoder with Retrieval-Token Speculation

This tutorial shows how to combine retrieval‑augmented generation with speculative decoding to cut latency while preserving factual grounding. You’ll build a runnable system that demonstrates real‑world performance gains.

A diagram of a KV cache with entries being evicted

Build a KV Cache Eviction Engine for LLM Context Windows

Implement a pluggable KV cache eviction engine from scratch, demonstrating systems thinking and production-ready patterns.

A code editor displaying Python implementation of paged attention and KV-cache management

Build a Minimal LLM Inference Engine with Paged Attention and Adaptive KV-Cache Eviction in Pure Python

A hands-on guide to building a minimal LLM inference engine with paged attention and adaptive KV-cache eviction in pure Python, designed to demonstrate real systems engineering prowess on your CV.

Diagram of a multi-GPU LLM inference pipeline with TensorRT-LLM INT8 quantization

Optimizing TensorRT-LLM: INT8 Quantization Pipelines for Real-Time Inference on Multi-GPU Clusters

A practical guide to INT8 quantization in TensorRT-LLM, covering pipeline architecture, kernel-level optimizations, and multi-GPU deployment strategies for real-time serving.