Cv | martinuke0's Blog

A Pure‑Python Continuous‑Batching Inference Engine for LLM Requests

A hands‑on guide to building a pure‑Python continuous‑batching inference engine that packs variable‑length LLM requests into GPU‑friendly batches with adaptive timeout and priority scheduling.

Mixture of experts routing layer illustration

From‑Scratch Mixture‑of‑Experts Routing Layer in Pure Python

A step‑by‑step tutorial creating a from‑scratch MoE routing layer in pure Python, complete with code, tests, and extension ideas for your CV.

Int4-quantized language model on a laptop

Hands-On Int4 Quantization Portfolio: Build a GPTQ/AWQ-Powered LLM CV Project

Learn how to quantize a language model to int4, integrate GPTQ and AWQ, and ship a runnable CV project that impresses hiring managers.

Hands‑On: Building a Zero‑Dependency Pure‑Python GGUF Model Loader with Memory‑Mapped Tensor Access

A lightweight, dependency‑free Python loader for GGUF models that uses memory‑mapped files and on‑the‑fly dtype conversion, perfect for a standout CV project.

Python code on a laptop screen with neural network diagrams

Hands‑On Build Guide: Speculative Decoding Inference Engine in Python

A step‑by‑step tutorial to build a speculative decoding inference engine in pure Python, demonstrating model parallelism, benchmarking, and production‑grade extensions.