CodePark/ questions / Monitoring, lag & scaling consumers
START PRACTICINGPRACTICE
Backend question · Message queues & streaming › Monitoring, lag & scaling consumers

How does the SQS visibility timeout work, and what goes wrong if it is shorter than your processing time?

When a consumer receives a message, SQS hides it from other consumers for a set time. This time is the visibility timeout. It is 30 seconds by default, and up to 12 hours. SQS does not lock a message to one consumer forever.

The consumer must delete the message with DeleteMessage after processing it. If it does not delete it in time, the message becomes visible again. SQS then delivers it to another consumer. This is how SQS recovers from crashed consumers. It is also why SQS is at-least-once: a message can arrive more than once.

If processing takes longer than the timeout, things go wrong:

  • A second consumer gets the same message while the first is still working on it.
  • The message is processed twice, and both consumers may try to delete it.
  • Each receive increases the message's receive count. This can push a healthy message into the DLQ (dead-letter queue: where messages go after too many failed tries).

Set the timeout well above your normal processing time. For long jobs, extend it while you work with ChangeMessageVisibility.

  • A received message is hidden for the visibility timeout, which is 30 s by default and 12 h at most.
  • The consumer must delete the message after processing it.
  • If it is not deleted in time, the message becomes visible again and is delivered again.
  • A timeout that is too short causes duplicate processing and higher receive counts.
  • For long jobs, extend the timeout with ChangeMessageVisibility.