CodePark/ questions / Kafka: topics, partitions & brokers
START PRACTICINGPRACTICE
Backend question · Message queues & streaming › Kafka: topics, partitions & brokers

How does Apache Kafka 4.0 store cluster metadata such as topics, partitions and leaders?

In KRaft mode: a Raft-based quorum of controller nodes; ZooKeeper support has been removed

Kafka 4.0 uses KRaft mode. KRaft replaces ZooKeeper with a group of Kafka controller nodes that agree on metadata using the Raft protocol. This group is called a quorum.

The history:
• KRaft became production-ready in the 3.x versions.
• ZooKeeper mode was then deprecated.
• Kafka 4.0 removed ZooKeeper completely.

So "In a ZooKeeper ensemble, which is still required" is wrong for 4.0. The __consumer_offsets topic is also wrong: it stores committed consumer offsets, not cluster metadata.