CodePark/ questions / Message queues & async work
START PRACTICINGPRACTICE
Backend question · System design & architecture › Message queues & async work

A Kafka topic has 6 partitions and a consumer group of 6 instances. Events for the same order must be processed in order. What should you do?

Use the order id as the message key, so all of an order's events go to the same partition

Kafka only keeps messages in order inside one partition. The default partitioner sends all messages with the same key to the same partition. So if you use the order id as the key, each order's events stay in sequence. Different orders still go to different partitions, so they are processed in parallel.

"Reduce the topic to a single partition" and "Run a single consumer instance in the group" also give you ordering. But they throw away all parallelism, so only one message is processed at a time.