Speculative-Decoding

Building a Speculative Decoding Engine From Scratch: A Portfolio Project That Actually Signals Systems Skill

Build a runnable speculative decoding engine in Python with an n-gram draft model and target-model verification loop — a portfolio project that demonstrates LLM inference, KV-cache reasoning, and acceptance sampling.

Diagram of a speculative decoder with Medusa draft heads feeding a verification pass on a target model.

Building a Speculative Decoder with Medusa-Style Draft Heads in PyTorch

Build a speculative decoder with Medusa-style draft heads and modern samplers in PyTorch. A production-flavored portfolio project that signals real ML systems skill.

Diagram of a speculative decoder: a small draft model proposing tokens that a larger target model verifies in parallel.

Build a Speculative Decoder From Scratch: A Portfolio Project That Signals Real Systems Skill

A working engineer’s guide to implementing a speculative decoder from scratch: draft with n-gram proposals, verify with a target model, ship it as a portfolio piece. Real code, real benchmarks, real extensions.

Diagram of a draft model proposing tokens and a target LLM verifying them in a speculative decoding loop.

Build a Speculative Decoding Sampler: A CV-Worthy Side Project

A complete, runnable Python project that pairs a draft n-gram model with a target LLM to verify multiple tokens per forward pass, plus a roadmap for turning it into production-flavored work.

Two pipelines merging — a fast draft model feeding into a verifier that accepts or rejects proposed tokens.

Speculative Decoding: How Production LLM Systems Cut Latency by 2–3x

Speculative decoding trades a small draft model for big latency wins — 2–3x faster token generation with mathematically identical outputs. Here’s how it works and where production systems use it.