CodePark/ questions / Queues vs streams vs pub/sub
START PRACTICINGPRACTICE
Backend question · Message queues & streaming › Queues vs streams vs pub/sub

When should you NOT put a message queue between two components?

Skip the queue when the caller needs the result right now to answer its own request. For example, checking a coupon during checkout is a request/response problem. If you do it over a queue, you must build extra parts to fake a synchronous call:

  • correlation ids, to match each reply to its request
  • reply queues
  • timeouts

Also skip the queue when the work is quick, cheap and happens inside one process. A queue has real costs:

  • a broker that you must run and monitor
  • at-least-once delivery, which means a message can arrive twice, so your handlers must be idempotent (safe to run twice)
  • eventual consistency: the result shows up later, not at once
  • harder debugging
  • retry handling and a DLQ (dead-letter queue: where failed messages go)

A queue is worth it when:

  • the work is slow or comes in bursts
  • the producer must keep working even when the consumer is down
  • several consumers need the same event
  • you need retries with backoff (waiting longer between each retry)

If none of these apply, a direct call or a normal function in the same process is simpler and easier to understand.

  • Don't use a queue when the caller needs the answer synchronously, right away.
  • Don't use a queue for quick, cheap work inside one process.
  • Queues cost extra infrastructure, duplicate messages, eventual consistency and harder debugging.
  • Use a queue for slow or bursty work, for independence from the consumer being up, for fan-out, or for retries.