TL;DR — A financial ledger is one of the few systems where being eventually consistent is unacceptable. This post walks through the architecture, invariants, and production patterns that keep money correct: double-entry as a non-negotiable invariant, event sourcing for auditability, idempotent posting for retries, and daily reconciliation as the last line of defense.

Why Ledgers Are Different

Most distributed systems optimize for availability and latency. A ledger optimizes for correctness first, then everything else. When a Kafka cluster drops a message, you can re-read and replay. When a ledger drops a debit, a customer is missing money, and the business learns about it from a chargeback.

This asymmetry changes every design decision:

  • Retries must be safe. The same posting cannot create two debits.
  • Partial failures must be reconcilable. If the journal write succeeds but the balance cache update fails, the system must converge.
  • Audit trails must be complete. Every state change needs provenance — who, when, why, with what reference.
  • Time is not monotonic. Backdated corrections, timezone shifts, and out-of-order arrival are normal.

A “regular” CRUD service can paper over these with retries and idempotency keys. A ledger treats them as first-class design concerns. As Martin Kleppmann’s work on consistency shows, distributed coordination is hard even when money is not involved; adding money makes every bug load-bearing.

The Core Invariant: Double-Entry

The single most important rule in any ledger is that every economic event moves money between accounts such that sum of debits equals sum of credits, per transaction. This is not an optimization; it is the invariant that lets you catch corruption the moment it happens.

A transfer of $100 from Account A (checking) to Account B (savings) is not one operation. It is two:

DR  checking    100.00
    CR  savings          100.00

If you record only one half, the books no longer balance — and that imbalance is exactly the signal an auditor (or a reconciliation job) will detect.

Account modeling

Accounts are identified by an immutable ID and a type (asset, liability, equity, revenue, expense). A practical representation:

CREATE TABLE accounts (
  id           UUID PRIMARY KEY,
  code         TEXT UNIQUE NOT NULL,        -- e.g. "1000-CASH"
  type         TEXT NOT NULL,               -- asset|liability|...
  currency     CHAR(3) NOT NULL,
  parent_id    UUID REFERENCES accounts(id),
  created_at   TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE TABLE postings (
  id           UUID PRIMARY KEY,
  txn_id       UUID NOT NULL REFERENCES transactions(id),
  account_id   UUID NOT NULL REFERENCES accounts(id),
  amount_minor BIGINT NOT NULL,              -- store cents, never floats
  direction    CHAR(1) NOT NULL CHECK (direction IN ('D','C')),
  posted_at    TIMESTAMPTZ NOT NULL
);

CREATE UNIQUE INDEX uniq_posting_per_side
  ON postings (txn_id, account_id, direction);

Two design notes worth flagging:

  1. Store money as integers (minor units). Floating point is a non-starter — 0.1 + 0.2 ≠ 0.3 and a balance column is exactly the place this bite you. Use BIGINT for cents or a decimal type if your database supports it well (PostgreSQL’s NUMERIC(20,4) is fine; MySQL’s DECIMAL works too).
  2. Per-currency accounts. A USD account and an EUR account are different ledgers. FX conversion is a transaction of its own, with explicit rates and dates. Don’t smuggle FX into a numeric column.

The balance invariant in code

A simple check that catches the vast majority of bugs:

def assert_balanced(postings: list[Posting]) -> None:
    total = sum(p.amount_minor * (1 if p.direction == 'D' else -1)
                for p in postings)
    if total != 0:
        raise UnbalancedTransactionError(
            f"Postings sum to {total} minor units, expected 0"
        )

Run this in three places: at the application boundary before commit, as a database CHECK constraint, and in a periodic background reconciler. Defense in depth is the only honest way to handle invariants that someone will eventually violate by accident.

Architectural Shapes

There are three shapes you’ll see in production, each with different tradeoffs.

Shape 1: Append-only journal + materialized balances

This is the classic accounting system. Every posting is written to an append-only journal. Balances are derived by summing the journal.

[Journal: append-only postings] ---> [Balance snapshots, updated async]
              |
              v
       [Reconciliation job]

Pros: perfect audit trail, easy to replay, trivial to add new account types. Cons: balance reads can be expensive on a busy account. The solution is materialized balance rows updated in the same transaction as the postings, refreshed from the journal on suspicion of drift.

This is the shape used by most modern fintechs, including Stripe’s ledger system and the design described in CockroachDB’s financial-services documentation.

Shape 2: Event-sourced transactions

Here the ledger is a stream of TransactionRecorded events. Each event is a self-contained, immutable description of a transfer. Current balances are derived by folding the stream.

@dataclass(frozen=True)
class TransactionRecorded:
    txn_id: UUID
    occurred_at: datetime
    idempotency_key: str
    postings: tuple[Posting, ...]

    def apply(self, balances: dict[AccountId, int]) -> dict[AccountId, int]:
        self.assert_balanced()
        next_ = balances.copy()
        for p in self.postings:
            sign = 1 if p.direction == 'D' else -1
            next_[p.account_id] = next_.get(p.account_id, 0) + sign * p.amount_minor
        return next_

This is the natural fit for Kafka-backed systems. The ledger is just a compacted topic keyed by account, and balances are KTable-style aggregations. Apache Kafka’s exactly-once semantics make this tractable, though “exactly once” really means “effectively once with idempotent producers and transactional writes.”

The catch: you must keep the stream forever, or you must snapshot balances and be willing to re-fold from snapshot. Most shops do both — keep the stream for N years for compliance, snapshot balances every night for fast recovery.

Shape 3: CRDT-style account heads

For high-volume wallets (ad credits, gaming currencies, IoT metering) where you don’t need strict double-entry semantics, you can model each account as a counter and reconcile with the journal asynchronously. The journal is still the source of truth; the heads are an optimization.

I list this for completeness, but be honest with yourself: if money leaves your system, you probably need Shape 1 or 2, not this one.

Idempotency: The Non-Negotiable

Network calls fail. A client retries. A worker times out and re-processes. If your posting handler is not idempotent, you will double-charge someone. This is not a theoretical risk — it is the single most common production bug in ledger systems.

The idempotency key pattern

Every external write request carries an Idempotency-Key header (UUID, generated by the client, persisted for at least 24 hours). The server:

  1. Inserts a row in idempotency_keys with the key and the result hash.
  2. If insert succeeds, processes the transaction.
  3. If insert fails (unique violation), returns the cached result.
CREATE TABLE idempotency_keys (
  key         TEXT PRIMARY KEY,
  request_hash BYTEA NOT NULL,         -- SHA-256 of normalized request
  result      JSONB NOT NULL,
  created_at  TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX idempotency_keys_created_at_idx
  ON idempotency_keys (created_at);

The request_hash check is important: if the same key arrives with a different payload, that’s a client bug and you should reject it. Stripe’s idempotency documentation describes the same pattern and is worth reading for the failure modes they enumerate.

Postgres-specific safeguards

If you run on Postgres, the classic race-condition pattern is:

INSERT INTO idempotency_keys (key, request_hash, result)
VALUES ($1, $2, $3)
ON CONFLICT (key) DO NOTHING
RETURNING key;

If RETURNING returns a row, you own the slot. If not, the request was already processed; fetch the cached result. Combined with SERIALIZABLE isolation on the transaction commit, this gives you exactly-once effect for free.

Concurrency Without Tears

The hardest operational problem in a ledger is two transactions touching the same account at the same time. Say a customer has $100 and submits two $80 transfers simultaneously. Without coordination, both might pass a balance check and you have a $60 phantom balance.

Pessimistic locking

SELECT balance FROM accounts
WHERE id = $1
FOR UPDATE;

Simple and correct, but creates contention. Fine for a small number of hot accounts; miserable for a payroll run on the same account.

Optimistic concurrency

UPDATE accounts
SET balance = balance - $1, version = version + 1
WHERE id = $2 AND version = $3;

If the affected row count is zero, the transaction lost a race. The application retries with a fresh read. This is the pattern that scales — Google describes it in the Spanner whitepaper as part of their general transaction model.

Hot-account sharding

For known hot spots (a major merchant’s settlement account, an exchange’s omnibus wallet), split the logical account into N physical shards. Debits and credits are routed across shards, and a background aggregator reconciles them into the logical balance. This is how crypto exchanges handle hot wallets, and the technique transfers cleanly to fiat.

The general principle: identify hot accounts before they bite you, and design the data model to let you shard them.

Reconciliation: The Last Line of Defense

Every production ledger has bugs. Reconciliation is how you find them before your customers do.

The three reconciliations you must run

  1. Internal balance check. Every account balance equals the sum of its postings. Run this hourly, every account, no exceptions. If it ever fails, page someone.

  2. External reconciliation. Compare your ledger against an external system of record — a bank statement, a card network file, a payment processor’s report. Run this daily, per currency. The variances are usually fees, FX, or timing, but sometimes they are bugs.

  3. Cross-system totals. The sum of all asset accounts must equal the sum of all liability and equity accounts — by definition. If it doesn’t, somewhere a double-entry transaction was split across two databases. (This is more common than you think when ledgers are spread across microservices.)

A pragmatic reconciler:

def reconcile_account(account_id: UUID) -> int:
    journaled = db.scalar("""
        SELECT COALESCE(SUM(CASE direction
                              WHEN 'D' THEN amount_minor
                              ELSE -amount_minor END), 0)
        FROM postings WHERE account_id = %s
    """, (account_id,))
    cached = db.scalar("SELECT balance_minor FROM account_balances "
                       "WHERE account_id = %s", (account_id,))
    return journaled - cached

If the diff is non-zero, alert. If it is non-zero on a settled account, page immediately.

Patterns in Production

A few patterns I’ve seen pay for themselves repeatedly.

Posting in the same transaction as the state change

Never update a balance and emit an event in two steps. Always do it in one database transaction:

BEGIN;
  -- insert postings
  INSERT INTO postings (...) VALUES (...);
  -- update balance
  UPDATE account_balances SET balance = balance + $delta ...
  -- write outbox row
  INSERT INTO outbox (event_type, payload) VALUES ('PostingCommitted', $payload);
COMMIT;

The outbox pattern — well-described in Microservices.io’s outbox pattern page — lets a separate process publish events to Kafka without the dual-write problem. Without this, you’ll see “the balance updated but the event was never published” bugs, and they are devastating.

Immutability by enforcement, not convention

Postings tables should be insert-only. Use database triggers or row-level security to block UPDATE and DELETE. Even superusers should have to go through an admin procedure to write a correction posting. The first time you have to do a forensic investigation, you’ll thank yourself.

Use a dedicated currency type

Whether it’s Postgres’s MONEY, a NUMERIC(20,4), or a domain type in your application layer, having a single representation of money prevents the “this field is in dollars and that field is in cents” bug. The cost of retrofitting is enormous; the cost of doing it right on day one is zero.

Backdated corrections

Real ledgers need to be corrected — typos, missing postings, regulatory restatements. The pattern is a reversing entry, not a destructive edit. Original postings stay forever; a paired set of reversing postings sets the balance back to where it should be, with a corrects_txn_id reference for traceability.

ALTER TABLE transactions
  ADD COLUMN corrects_txn_id UUID REFERENCES transactions(id),
  ADD COLUMN reason TEXT NOT NULL CHECK (length(reason) >= 10);

The CHECK (length(reason) >= 10) is small but valuable: it forces anyone making a correction to write a real explanation, which you will need when an auditor asks six months later.

What I’d Build Today

If I were designing a greenfield ledger for a fintech in 2026, I would:

  • Postgres as the system of record, with logical replication to a read replica for analytics.
  • Append-only journal of postings, with materialized balance rows updated in the same transaction.
  • Outbox table for emitting PostingCommitted events to Kafka, consumed by downstream services.
  • Idempotency-key table with a 72-hour retention window and request-hash validation.
  • Hourly reconciliation that diffs journal totals against cached balances, paging on any non-zero result.
  • Daily external reconciliation against bank and processor reports, with a small variance queue for ops to triage.
  • Strict double-entry enforcement at the database level via a CHECK constraint that asserts postings sum to zero per transaction.
  • A correction framework that requires a reason and produces reversing entries, never destructive edits.

The stack is unremarkable. The discipline is what keeps money correct.

Key Takeaways

  • Double-entry is a hard invariant, not a style choice. Enforce it in code, in the database, and in reconciliation. Three layers or it isn’t really enforced.
  • Idempotency keys are not optional. Every external write needs one, with a request-hash check to catch client bugs.
  • Postings and balance updates must be in the same transaction. Use the outbox pattern to publish events safely.
  • Money is integers in minor units. No floats, ever. Pick a representation and stick to it.
  • Reconcile hourly, externally daily, and cross-system weekly. The reconciler is your canary; if it ever fails, treat it as an incident.
  • Corrections are reversing entries, not edits. Immutability is what makes the audit trail trustworthy.
  • Hot accounts will bite you. Design your data model to allow sharding before you need it.

Further Reading