Producers are publishing faster than consumers can process, and the backlog keeps growing. What is back-pressure, and what are your options?

Back-pressure is a way for a slow part of the system to tell the parts before it: "I can't keep up." Then the load is slowed down or dropped. Without it, work piles up with no limit. In the end, memory runs out, or the delay becomes so long that the results are useless.

Your options, roughly in this order:

• Scale consumers if the bottleneck is consumer capacity. In Kafka, this only helps up to the number of partitions.
• Bound the buffers. Limit how much work each consumer has in progress at once, for example with RabbitMQ's prefetch count. Use queues with a maximum size.
• Slow down or reject producers when the queues are full. Block them, or return 429 or 503 with a Retry-After header at the API edge.
• Shed or degrade load. Drop low-priority work, process only a sample, or batch messages together.
• Separate queues by priority. Then a flood of bulk jobs cannot starve urgent ones.

Also watch queue depth (how many messages wait) and consumer lag (how far consumers are behind), with alerts. And ask whether old messages are still worth processing at all.