Benchmarking Throughput vs Latency

The throughput / latency / durability triangle

Throughput in Kafka is governed by:

• linger.ms — wait this long to fill a batch (latency cost, throughput win).
• batch.size — bigger batches → bigger savings per fsync.
• compression.type — zstd ≈ best ratio, lz4 ≈ best speed, snappy ≈ default sweet spot.
• acks — all halves throughput vs 1 (typically).

Production benchmarking belongs in a real cluster with kafka-producer-perf-test.sh and the in-app code below — both. Don't tune blind.

Sweep linger.ms ∈ {0, 5, 20} × RequiredAcks ∈ {Leader, All} and report msgs/s and p99 latency. Skeleton below — extend the sweep loop.