CodePark/ questions / Kafka: consumer groups & offsets
START PRACTICINGPRACTICE
Backend question · Message queues & streaming › Kafka: consumer groups & offsets

A topic has 4 partitions. You scale the consumer group to 6 instances. What happens?

4 instances get one partition each and 2 sit idle

4 instances get one partition each, and 2 instances sit idle. Inside one consumer group, Kafka gives each partition to at most one consumer. So the partition count is the limit on how many consumers in the group can work in parallel. The extra 2 instances get nothing.

The idle instances are still useful. If another member dies, they take over its partitions.


"Each partition is split between instances" is wrong, because one partition never has two consumers in the same group. "Kafka adds 2 partitions automatically" is also wrong: Kafka never adds partitions on its own.