martinuke0's Blog

A autonomous ride-hail vehicle navigating an urban street at dusk, with sensor housings visible on the roof.

How Cybercabs Work: The Engineering Behind Autonomous Ride-Hail

Cybercabs are not magic. They are a tightly integrated stack of perception, prediction, planning, and fleet operations software wrapped around an electric vehicle. This post walks through each layer with the production systems that ship it today.

Abstract visualization of a GPU pipeline routing inference requests across multiple LLM instances.

Serving LLMs in Production: A Practical Guide to Latency, Throughput, and Cost

A production-focused walkthrough of how to serve large language models efficiently, covering batching strategies, quantization, KV cache management, and the observability you actually need.

Two conveyor belts running at different speeds, one fast and one slow, photographed from above.

Throughput vs Latency: Why Optimizing One Will Almost Always Break the Other

Throughput and latency measure different things, and most of the systems pain we feel at scale comes from pretending they don’t. Here is how they actually trade off, with real production patterns.

Stylized illustration of stacked HBM dies next to a GPU die on an interposer.

GPU Memory in 2026: HBM, KV-Cache, and the Real Bottlenecks Behind LLM Inference

A working engineer’s guide to GPU memory: HBM bandwidth, KV-cache math, fragmentation, and the production patterns that keep large model inference honest.

Two pipelines merging — a fast draft model feeding into a verifier that accepts or rejects proposed tokens.

Speculative Decoding: How Production LLM Systems Cut Latency by 2–3x

Speculative decoding trades a small draft model for big latency wins — 2–3x faster token generation with mathematically identical outputs. Here’s how it works and where production systems use it.