AWS SQS & SNS Support
SQS visibility timeout and duplicate messages.
When a consumer receives an SQS message, the message is hidden for the visibility timeout. If processing takes longer, the message reappears and a second consumer processes it. Standard queues can also deliver a message more than once. We fix the timeout and make the consumer safe to repeat.
When You Need This
The situations where this work pays for itself.
Signs it is time
- The same message is processed two or more times
- Jobs run twice when they take longer than expected
- A Lambda consumer shows errors and repeats work
- Messages reappear in the queue while still being handled
- Duplicate emails, charges or database rows appear
What you get
- Visibility timeout settings with the measurements behind them
- Idempotency implementation in the consumer
- A heartbeat pattern for long jobs
- Alarms for message age and in-flight counts
How we do it
A clear, safe process you can follow along with.
1. Measure how long processing takes
We compare p99 processing time with the queue's visibility timeout using consumer logs and the ApproximateAgeOfOldestMessage and ApproximateNumberOfMessagesNotVisible metrics.
2. Set the timeout above processing time
Visibility timeout should exceed the longest normal processing time with margin. For Lambda consumers, AWS recommends setting it to at least six times the function timeout so retries do not cause early redelivery.
3. Extend for variable-length jobs
For workers with variable duration we call ChangeMessageVisibility as a heartbeat to extend the timeout while work continues, up to the maximum allowed from the original receive.
4. Delete messages only on success
We confirm the consumer deletes a message after the work is committed, not before, and that errors do not delete it. Deleting too early loses work, too late causes repeats.
5. Make processing idempotent
Because standard queues are at-least-once, duplicates can happen even with a good timeout. We use an idempotency key per message, such as a conditional write to DynamoDB, or Powertools for AWS Lambda idempotency utilities.
6. Consider FIFO where order or dedupe is needed
FIFO queues deduplicate by message deduplication ID within the deduplication interval, but add throughput and ordering considerations. We recommend it only where it fits.
Mistakes we help you avoid
Common ways this goes wrong, and what we do instead.
Leaving the default 30 seconds
Any job over 30 seconds is delivered again.
Making the timeout huge to hide the problem
A long timeout delays retry of real failures.
Assuming exactly-once on standard queues
Duplicates are possible by design. Handle them.
Deleting before the side effect is done
A crash then loses the message permanently.
Frequently Asked Questions
What is the SQS visibility timeout?
It is the period after a message is received during which other consumers cannot see it. If the message is not deleted in time, it becomes visible again.
Why do I get duplicate SQS messages?
Either processing outlasts the visibility timeout, or a standard queue delivered the message more than once, which can happen. Idempotent consumers handle both.
What visibility timeout should Lambda use?
At least the function timeout, and AWS guidance suggests six times the function timeout plus the batching window. We confirm against current documentation.
Want Help With SQS visibility timeout and duplicate messages?
Schedule an appointment. Tell us about your environment and we will outline the safest, highest-value approach.
Schedule Appointment › No commitment required