The Log as Source of Truth

An append-only log + offsets = time travel for your data.

0/3 done

Append-only is a superpower

The log is the database

Kafka's central abstraction is the partitioned, immutable, append-only log. Producers append; consumers read by offset. Because writes never mutate prior records, the log is:

  • a replayable history (start from offset 0 to rebuild any derived state),
  • a decoupling layer (N producers, M consumers, zero awareness of each other),
  • a time machine (commit logs back every event-sourced system worth shipping).

Compare to a row in a relational DB: every UPDATE destroys information.

Analogy

An accountant's ledger never lets you erase a row — you write a compensating entry. That's why auditors trust it. A Kafka topic is the same ledger, but the auditors are your downstream services.

Reading in progress · 0 of 3 activities done