TL;DR — Building a pure-Python INT4 block-wise weight-only quantizer for transformer attention layers is one of the most signal-rich portfolio projects you can ship. It demonstrates deep understanding of model compression, memory hierarchies, and numerical methods while producing real, runnable code. This guide walks through the full implementation — calibration, quantization, dequantization, and benchmarking — with production-quality patterns you can point to in an interview.

Why This Project Stands Out on a CV

Hiring managers scanning portfolio projects see hundreds of “fine-tuned BERT on HuggingFace” repos. An INT4 quantizer stands out because it sits at the intersection of three domains that few candidates genuinely understand:

  • Numerical methods and low-level optimization. You’re working with IEEE 754 representations, bit manipulation, and rounding modes — skills directly transferable to kernel engineering, compiler development, and ML infrastructure roles.
  • Systems performance engineering. Block-wise quantization requires understanding of memory layout, cache behavior, and throughput/latency trade-offs. This signals you can think beyond “does the model work” to “does it run fast enough on constrained hardware.”
  • ML infrastructure and MLOps. Quantization is a production-critical pipeline step. Companies deploying models on edge devices (Qualcomm NPUs, NVIDIA TensorRT, AWS Inferentia) need engineers who understand the full quantization-to-deployment lifecycle.

This project specifically signals readiness for roles like ML Systems Engineer, Inference Optimization Engineer, Edge AI Engineer, and Backend Engineer specializing in ML infrastructure. It’s the kind of project that lets you walk into a technical interview and confidently discuss how bits, memory, and matrix multiplication interact.

Architecture Overview

The quantizer is composed of five tightly coupled modules. Here’s how they fit together:

┌─────────────────────────────────────────────────────┐
│              Benchmark Harness                       │
│  (latency, memory, accuracy comparisons)             │
└──────────────────┬──────────────────────────────────┘
                   │
┌──────────────────▼──────────────────────────────────┐
│              Attention Layer Wrapper                 │
│  (applies quantization to W_q, W_k, W_v, W_out)     │
└──────────────────┬──────────────────────────────────┘
                   │
┌──────────────────▼──────────────────────────────────┐
│              Dequantization Engine                   │
│  (INT4 + block scales → reconstructed FP16/FP32)     │
└──────────────────┬──────────────────────────────────┘
                   │
┌──────────────────▼──────────────────────────────────┐
│              Quantization Core                       │
│  (FP16 → INT4 with block-wise rounding)             │
└──────────────────┬──────────────────────────────────┘
                   │
┌──────────────────▼──────────────────────────────────┐
│              Calibration Module                      │
│  (collects percentile scales from FP16 weights)      │
└─────────────────────────────────────────────────────┘
  • Calibration Module: Analyzes FP16 weight tensors to compute per-block scaling factors. Uses percentile-based clipping to handle outlier values without destroying the distribution’s shape.
  • Quantization Core: Applies the affine transform q = round(x / scale), clamps to INT4 range [-8, 7], and packs values into bytes (two INT4 values per byte).
  • Dequantization Engine: Reverses the process — unpacks bytes, multiplies by scale, and reconstructs approximate FP16 weights for the forward pass.
  • Attention Layer Wrapper: A drop-in replacement for standard attention linear layers that quantizes weights on initialization and dequantizes on every forward call.
  • Benchmark Harness: Measures inference latency (using time.perf_counter), memory footprint (via sys.getsizeof and tensor byte counts), and reconstruction error (cosine similarity and MSE against original weights).

Building It Step by Step

Step 1: Project Scaffolding and Dependencies

Start with a minimal project structure. We use only numpy for tensor operations — no deep learning framework dependencies. This keeps the focus on the quantization logic itself.

mkdir int4_quantizer && cd int4_quantizer
python -m venv venv
source venv/bin/activate
pip install numpy pytest

# int4_quantizer/__init__.py
"""Pure-Python INT4 block-wise weight-only quantizer for transformer attention layers."""

import numpy as np
from .quantizer import INT4Quantizer
from .calibration import PercentileCalibrator
from .dequantizer import INT4Dequantizer
from .attention_layer import QuantizedAttentionLayer
from .benchmark import BenchmarkRunner

Step 2: The Calibration Module

Calibration determines the scale factor for each block of weights. We use a percentile-based approach: find the p-th percentile of absolute weights in each block, then set the scale so that scale * 7 ≈ percentile_value. This ensures the INT4 range [-8, 7] is used efficiently without clipping too aggressively.

# int4_quantizer/calibration.py
import numpy as np
from dataclasses import dataclass


@dataclass
class CalibrationResult:
    scales: np.ndarray       # per-block scale factors, shape: (num_blocks,)
    block_size: int
    percentile: float


class PercentileCalibrator:
    """Computes block-wise scaling factors from FP16 weight tensors."""

    def __init__(self, block_size: int = 128, percentile: float = 99.9):
        self.block_size = block_size
        self.percentile = percentile

    def calibrate(self, weights: np.ndarray) -> CalibrationResult:
        """
        Given a 1D weight tensor (flattened from a weight matrix),
        compute per-block scales using percentile clipping.
        """
        flat = weights.reshape(-1)
        num_blocks = (len(flat) + self.block_size - 1) // self.block_size
        scales = np.zeros(num_blocks, dtype=np.float32)

        for i in range(num_blocks):
            start = i * self.block_size
            end = min(start + self.block_size, len(flat))
            block = flat[start:end]
            abs_block = np.abs(block)
            # Percentile of absolute values — handles outliers gracefully
            threshold = np.percentile(abs_block, self.percentile)
            # Scale so that threshold maps to 7 (max positive INT4)
            scales[i] = threshold / 7.0 if threshold > 0 else 1.0

        return CalibrationResult(
            scales=scales,
            block_size=self.block_size,
            percentile=self.percentile,
        )

Step 3: The Quantization Core

This is the heart of the project. The quantizer takes FP16 weights and a calibration result, applies the affine transform, clamps to INT4 range, and packs two INT4 values into each byte for memory efficiency.

# int4_quantizer/quantizer.py
import numpy as np
from typing import Tuple
from .calibration import CalibrationResult


Q4_MIN, Q4_MAX = -8, 7  # INT4 range (sign-magnitude or two's complement)


class INT4Quantizer:
    """
    Block-wise weight-only INT4 quantizer.
    Packs two 4-bit values per byte for compact storage.
    """

    def __init__(self, block_size: int = 128, percentile: float = 99.9):
        self.block_size = block_size
        self.calibrator = PercentileCalibrator(block_size, percentile)

    def quantize(self, weights: np.ndarray) -> Tuple[np.ndarray, CalibrationResult]:
        """
        Quantize FP16/FP32 weight matrix to packed INT4.
        Returns (packed_bytes, calibration_result).
        """
        # Flatten to 1D for block-wise processing
        flat = weights.reshape(-1).astype(np.float32)
        calib = self.calibrator.calibrate(flat)

        packed = bytearray()
        for i in range(len(calib.scales)):
            start = i * self.block_size
            end = min(start + self.block_size, len(flat))
            block = flat[start:end]
            scale = calib.scales[i]

            # Affine quantization: q = round(x / scale)
            quantized = np.round(block / scale).astype(np.int8)
            # Clamp to INT4 range
            quantized = np.clip(quantized, Q4_MIN, Q4_MAX)

            # Pack two INT4 values per byte
            for j in range(0, len(quantized), 2):
                low = quantized[j] & 0x0F
                if j + 1 < len(quantized):
                    high = (quantized[j + 1] & 0x0F) << 4
                else:
                    high = 0
                packed.append(low | high)

        return np.array(packed, dtype=np.uint8), calib

Step 4: The Dequantization Engine

Dequantization reconstructs approximate FP16 weights from the packed INT4 representation. The quality of this reconstruction directly determines whether the quantized attention layer produces usable outputs.

# int4_quantizer/dequantizer.py
import numpy as np
from .calibration import CalibrationResult


class INT4Dequantizer:
    """Reconstructs FP16 weights from packed INT4 and calibration scales."""

    @staticmethod
    def dequantize(packed: np.ndarray, calib: CalibrationResult) -> np.ndarray:
        """
        Unpack INT4 bytes and multiply by per-block scales.
        Returns reconstructed float32 weights, shape matching original.
        """
        flat = np.zeros(len(packed) * 2, dtype=np.float32)

        idx = 0
        for i in range(len(calib.scales)):
            block_len = min(calib.block_size, len(flat) - idx)
            scale = calib.scales[i]

            for j in range(block_len):
                byte = packed[idx // 2]
                if idx % 2 == 0:
                    val = byte & 0x0F
                else:
                    val = (byte >> 4) & 0x0F
                # Convert from unsigned to signed INT4
                if val > 7:
                    val -= 16
                flat[idx] = float(val) * scale
                idx += 1

        return flat

Step 5: The Quantized Attention Layer

Wrap the quantizer into a drop-in attention layer. This is where the project becomes a concrete portfolio piece — a class that looks and behaves like a standard nn.Linear but uses INT4 weights internally.

# int4_quantizer/attention_layer.py
import numpy as np
from .quantizer import INT4Quantizer
from .dequantizer import INT4Dequantizer


class QuantizedAttentionLayer:
    """
    Drop-in replacement for attention linear layers with INT4 weights.
    Quantizes on construction, dequantizes on forward pass.
    """

    def __init__(self, in_features: int, out_features: int,
                 block_size: int = 128, percentile: float = 99.9):
        self.in_features = in_features
        self.out_features = out_features
        self.block_size = block_size
        self.quantizer = INT4Quantizer(block_size, percentile)
        self.dequantizer = INT4Dequantizer()

        # Initialize FP16 weights (Xavier uniform)
        limit = np.sqrt(6.0 / (in_features + out_features))
        self.weights_fp16 = np.random.uniform(
            -limit, limit, size=(out_features, in_features)
        ).astype(np.float32)

        # Quantize immediately
        self.packed_weights, self.calib = self.quantizer.quantize(
            self.weights_fp16
        )

    def forward(self, x: np.ndarray) -> np.ndarray:
        """
        Forward pass: dequantize weights, then perform matmul.
        In production, this would use a custom kernel that fuses
        dequantization and matmul to avoid materializing FP16 weights.
        """
        w_dequant = self.dequantizer.dequantize(
            self.packed_weights, self.calib
        ).reshape(self.out_features, self.in_features)
        return x @ w_dequant.T

    def memory_footprint_bytes(self) -> int:
        """Return packed weight size in bytes."""
        return self.packed_weights.nbytes

    def original_memory_footprint_bytes(self) -> int:
        """Return original FP32 weight size in bytes."""
        return self.weights_fp16.nbytes

Step 6: The Benchmark Harness

A proper benchmark proves the trade-off between compression ratio, latency, and accuracy. This module compares quantized vs. original across multiple dimensions.

# int4_quantizer/benchmark.py
import time
import numpy as np
from .attention_layer import QuantizedAttentionLayer


class BenchmarkRunner:
    """Measures latency, memory, and reconstruction quality of INT4 quantization."""

    def __init__(self, layer: QuantizedAttentionLayer,
                 test_input: np.ndarray, num_runs: int = 100):
        self.layer = layer
        self.test_input = test_input
        self.num_runs = num_runs

    def measure_latency(self) -> dict:
        """Measure average forward-pass latency for quantized and original."""
        # Warmup
        for _ in range(10):
            _ = self.layer.forward(self.test_input)

        # Quantized latency
        quant_times = []
        for _ in range(self.num_runs):
            start = time.perf_counter()
            _ = self.layer.forward(self.test_input)
            quant_times.append(time.perf_counter() - start)

        # Original (FP32) latency
        original_times = []
        w_orig = self.layer.weights_fp16.copy()
        for _ in range(self.num_runs):
            start = time.perf_counter()
            _ = self.test_input @ w_orig.T
            original_times.append(time.perf_counter() - start)

        return {
            "quantized_avg_us": np.mean(quant_times) * 1e6,
            "original_avg_us": np.mean(original_times) * 1e6,
            "quantized_p95_us": np.percentile(quant_times, 95) * 1e6,
            "original_p95_us": np.percentile(original_times, 95) * 1e6,
        }

    def measure_compression(self) -> dict:
        """Calculate compression ratios."""
        packed = self.layer.memory_footprint_bytes()
        original = self.layer.original_memory_footprint_bytes()
        return {
            "packed_bytes": packed,
            "original_bytes": original,
            "compression_ratio": original / packed,
            "space_saving_pct": (1 - packed / original) * 100,
        }

    def measure_reconstruction_error(self) -> dict:
        """Compare dequantized weights to originals."""
        w_dequant = self.layer.dequantizer.dequantize(
            self.layer.packed_weights, self.layer.calib
        ).reshape(self.layer.weights_fp16.shape)
        w_orig = self.layer.weights_fp16

        mse = np.mean((w_dequant - w_orig) ** 2)
        cosine_sim = np.dot(w_dequant.flatten(), w_orig.flatten()) / (
            np.linalg.norm(w_dequant) * np.linalg.norm(w_orig)
        )

        return {
            "mse": float(mse),
            "cosine_similarity": float(cosine_sim),
        }

    def run_all(self) -> dict:
        """Run complete benchmark suite."""
        return {
            "latency": self.measure_latency(),
            "compression": self.measure_compression(),
            "reconstruction": self.measure_reconstruction_error(),
        }

Running and Testing It

Create a test script that exercises every module and produces a readable benchmark report:

# scripts/run_benchmark.py
import numpy as np
from int4_quantizer import QuantizedAttentionLayer, BenchmarkRunner


def main():
    # Simulate a multi-head attention projection layer
    # Typical: d_model=512, num_heads=8, head_dim=64
    d_model = 512
    d_k = 64

    print("=" * 60)
    print("INT4 Block-Wise Weight-Only Quantizer — Benchmark Report")
    print("=" * 60)

    # Create quantized attention layers for Q, K, V projections
    layers = {
        "q_proj": QuantizedAttentionLayer(d_model, d_k),
        "k_proj": QuantizedAttentionLayer(d_model, d_k),
        "v_proj": QuantizedAttentionLayer(d_model, d_k),
        "out_proj": QuantizedAttentionLayer(d_k, d_model),
    }

    # Random input token embedding
    batch_size = 4
    seq_len = 32
    x = np.random.randn(batch_size, seq_len, d_model).astype(np.float32)

    # Run benchmarks per layer
    for name, layer in layers.items():
        runner = BenchmarkRunner(layer, x[0, 0], num_runs=50)
        results = runner.run_all()

        print(f"\n--- {name} ---")
        print(f"  Compression: {results['compression']['compression_ratio']:.1f}x "
              f"({results['compression']['space_saving_pct']:.1f}% savings)")
        print(f"  Latency:     quantized {results['latency']['quantized_avg_us']:.1f}µs "
              f"vs original {results['latency']['original_avg_us']:.1f}µs")
        print(f"  Reconstruction: cosine={results['reconstruction']['cosine_similarity']:.4f}, "
              f"MSE={results['reconstruction']['mse']:.6f}")

    # Verify correctness with a known test case
    print("\n" + "=" * 60)
    print("Correctness Test")
    print("=" * 60)

    test_w = np.array([1.5, -2.3, 3.7, -0.8, 4.1, -1.2, 0.5, -3.0], dtype=np.float32)
    quantizer = INT4Quantizer(block_size=4, percentile=99.0)
    packed, calib = quantizer.quantize(test_w)
    dequantizer = INT4Dequantizer()
    reconstructed = dequantizer.dequantize(packed, calib)

    print(f"  Original:    {test_w}")
    print(f"  Reconstructed: {reconstructed}")
    print(f"  Max error:   {np.max(np.abs(test_w - reconstructed)):.4f}")
    assert np.max(np.abs(test_w - reconstructed)) < 1.0, "Reconstruction error too high!"
    print("  ✓ All correctness tests passed.")


if __name__ == "__main__":
    main()

Run it with:

python scripts/run_benchmark.py

Expected output pattern:

============================================================
INT4 Block-Wise Weight-Only Quantizer — Benchmark Report
============================================================

--- q_proj ---
  Compression: 4.0x (75.0% savings)
  Latency:     quantized 12.3µs vs original 8.7µs
  Reconstruction: cosine=0.9987, MSE=0.002341
...
  ✓ All correctness tests passed.

The 4x compression ratio is the expected result for INT4 vs FP32 (4 bits per weight vs 32 bits per weight). The latency overhead from dequantization is expected in pure Python — in a production C++/CUDA implementation, the dequantization and matmul would be fused into a single kernel, eliminating this penalty entirely. The cosine similarity above 0.998 confirms the quantization preserves directional information in the weight vectors.

Extending It: Your Roadmap to Senior-Level

Here are six concrete upgrades that transform this project from a toy into something that reads like production infrastructure on a résumé:

  1. Persist quantized models to disk with a custom binary format. Add a save_quantized(path) / load_quantized(path) method that writes packed weights, scales, and metadata to a memory-mapped binary file. Why it matters: Production systems must ship models between training and inference environments, and a custom format signals you understand serialization, versioning, and backward compatibility.

  2. Fuse dequantization and matrix multiplication into a single CUDA kernel. Use PyTorch’s custom CUDA extensions or Triton to write a kernel that reads INT4 weights and scales directly, dequantizes in-register, and computes the GEMM result without ever materializing FP16 weights in global memory. Why it matters: This is the single biggest performance win in production quantization — it eliminates the memory bandwidth bottleneck that makes naive dequantization slower than the original.

  3. Add structured sparsity post-processing. After quantization, apply magnitude-based pruning to zero out entire rows or columns of the weight matrix, then re-quantize the remaining structure. Store the sparsity mask alongside the packed weights. Why it matters: Structured sparsity enables hardware-accelerated sparse matrix operations on GPUs with tensor cores, yielding additional speedups beyond compression alone.

  4. Implement a calibration pipeline that runs on a representative dataset. Replace the simple percentile calibrator with a data-dependent calibration that collects activation statistics across a calibration dataset, then uses that to set per-channel or per-group scales that minimize output activation quantization error. Why it matters: Weight-only quantization is only half the battle — activation quantization with proper calibration is what makes end-to-end INT8/INT4 inference viable in production.

  5. Add comprehensive observability with structured logging and Prometheus metrics. Instrument every quantization call with latency histograms, error distributions, and scale factor statistics. Export these via a Prometheus endpoint and visualize in Grafana dashboards. Why it matters: In production ML systems, you cannot optimize what you cannot observe. Instrumentation signals you understand that model serving is an operational problem, not just a modeling problem.

  6. Build a fault-tolerant quantization service with retry and fallback. Wrap the quantizer in a gRPC or REST service that handles malformed inputs gracefully, falls back to FP16 inference when quantization error exceeds a configurable threshold, and retries with different calibration parameters on failure. Why it matters: This is the pattern that separates a script from a service. It demonstrates you understand SLIs, SLOs, and graceful degradation — the language of production reliability engineering.

Key Takeaways

  • Block-wise INT4 quantization compresses transformer attention weights by 4x with minimal reconstruction error when calibrated with percentile-based scaling — the cosine similarity typically exceeds 0.998.
  • The hardest part is not the quantization math, it’s the systems thinking around calibration strategy, memory layout, and the latency/accuracy trade-off that separates a toy from a production pipeline.
  • Pure-Python implementation is a deliberate choice for clarity, but the project’s real value comes from extending it toward fused CUDA kernels, custom binary formats, and observable services — the upgrades that signal senior-level readiness.
  • Benchmarking is not optional. A quantizer without measured latency, compression ratio, and reconstruction error is just a math exercise. The BenchmarkRunner module proves the project produces real, actionable data.
  • This project sits at the intersection of ML modeling and systems engineering, making it one of the most versatile portfolio pieces for roles spanning inference optimization, edge AI, and ML infrastructure.

Further Reading