4 instances get one partition each and 2 sit idle
4 instances get one partition each, and 2 instances sit idle. Inside one consumer group, Kafka gives each partition to at most one consumer. So the partition count is the limit on how many consumers in the group can work in parallel. The extra 2 instances get nothing.
The idle instances are still useful. If another member dies, they take over its partitions.
"Each partition is split between instances" is wrong, because one partition never has two consumers in the same group. "Kafka adds 2 partitions automatically" is also wrong: Kafka never adds partitions on its own.
Read more
- Kafka consumers (https://developer.confluent.io/courses/apache-kafka/consumers/)developer.confluent.io
- Kafka introduction (https://kafka.apache.org/intro)kafka.apache.org