Quantization

A visualization of INT4 weight tensors being dequantized and multiplied in a fused GEMM kernel

Build a Blockwise INT4 Inference Engine: A Hands-On Portfolio Project

A hands-on build guide for a blockwise INT4 inference engine with packed-nibble weights, per-group scales/zero-points, and fused dequantizing GEMM — a portfolio project that demonstrates real systems and ML engineering depth.

A visualization of INT4 quantization layers inside a transformer model, showing weight matrices being compressed and dequantized.

Building a From-Scratch Group-Wise INT4 Quantization Layer for LLaMA Weights

A hands-on guide to implementing group-wise INT4 quantization for LLaMA-style transformers, covering per-group scale calibration, fused dequantization kernels, and benchmarks that demonstrate real ML-systems engineering skill.

Diagram of a multi-GPU LLM inference pipeline with TensorRT-LLM INT8 quantization

Optimizing TensorRT-LLM: INT8 Quantization Pipelines for Real-Time Inference on Multi-GPU Clusters

A practical guide to INT8 quantization in TensorRT-LLM, covering pipeline architecture, kernel-level optimizations, and multi-GPU deployment strategies for real-time serving.

Build a GGUF Parser with Memory-Mapped Quantized Tensor Loading in Pure Python

Build a production-grade GGUF parser from scratch in pure Python, featuring memory-mapped quantized tensor loading. Learn the format internals, write real dequantization kernels, and discover why this project stands out to hiring managers.

A diagram of a transformer model with highlighted KV cache

Building a Pure‑Python Int4 KV‑Cache for Transformer Inference

This project demonstrates low‑level memory optimization, quantization, and efficient inference techniques that signal strong systems engineering expertise to hiring managers.