Data-Engineering

Debezium CDC pipeline architecture diagram showing data flow between databases, Kafka, and sinks

Implementing Debezium CDC Pipelines: Exactly-Once State, Schema Evolution, and Backfills

Learn how to implement Debezium-based change data capture pipelines that guarantee exactly-once state, handle schema evolution gracefully, and execute reliable backfills at scale.

A dashboard showing personal data visualizations

Building a Personal Event-Driven Data Pipeline: A Portfolio Project for Systems Engineers

Learn to build a scalable, event-driven personal data pipeline using FastAPI, Kafka, and PostgreSQL — a portfolio project that signals senior-level systems thinking.

Diagram showing PostgreSQL WAL flowing through Debezium into Kafka topics and downstream sinks.

Architecting Change Data Capture with Debezium and PostgreSQL: A Production Pipeline Walkthrough

How to design a reliable CDC pipeline from PostgreSQL to downstream systems using Debezium — covering WAL internals, connector configuration, schema evolution, and the failure modes you’ll hit in production.

Diagram of a RAG ingestion pipeline showing document parsing, chunking, embedding, and indexing stages.

The Hardest RAG Ingestion Problems and How Engineers Actually Solve Them

Most RAG failures start long before the LLM is ever called. This post walks through the five hardest ingestion problems and the patterns teams use to fix them in production.

Probability distribution curves and a die resting on a notebook.

Probabilities Zero to Hero: From Coin Flips to Bayesian Inference in Production

A practical, systems-oriented walkthrough of probability concepts working engineers actually use, anchored in fraud detection, search ranking, and Bayesian A/B testing pipelines.