AWS SQS & SNS Support
Tuning Lambda SQS event source mappings.
A Lambda SQS trigger has a handful of settings that decide throughput, cost and failure behavior. Defaults often retry a whole batch because of one bad message, or scale faster than a database can handle. We tune batch size, partial batch responses and concurrency to fit your workload.
When You Need This
The situations where this work pays for itself.
Signs it is time
- One bad message causes a whole batch to be retried and reprocessed
- Lambda scales up and overloads a database or API
- Functions run with tiny batches and cost more than needed
- Messages end up in the DLQ even though most of the batch succeeded
- Queue backlog grows during bursts
What you get
- Event source mapping settings in infrastructure as code
- A handler updated for partial batch responses
- Metrics and alarms for queue age, errors and throttles
- Test results for failure and burst scenarios
How we do it
A clear, safe process you can follow along with.
1. Review current behavior
We read the event source mapping settings and the metrics: ApproximateAgeOfOldestMessage, Lambda Duration, Errors, Throttles and ConcurrentExecutions, to see whether the problem is speed, cost or failures.
2. Enable partial batch responses
With ReportBatchItemFailures set in FunctionResponseTypes, the function returns batchItemFailures listing only failed message IDs. Without it, any error returns the full batch to the queue and successful messages are reprocessed.
3. Set batch size and window
Larger batches lower the invocation count, but raise duration per invocation and the cost of a failure. We choose the batch size from per-message processing time, and use a batching window when throughput is low and cost matters.
4. Cap concurrency
The mapping's maximum concurrency setting limits how many concurrent invocations the queue can drive, protecting downstream systems. It has a minimum value, so check current documentation. Reserved concurrency on the function is another control.
5. Align timeouts and the DLQ
The queue's visibility timeout must exceed the function timeout, with margin. The DLQ belongs on the source queue, not only on the function, and maxReceiveCount must allow for transient retries.
6. Test with a poison message and a burst
We send a mix of good and bad messages and a large backlog, then check that only failures retry, concurrency stays inside limits and the backlog drains in the time you need.
Mistakes we help you avoid
Common ways this goes wrong, and what we do instead.
Throwing an error for one bad message
Without partial batch responses the whole batch retries.
Visibility timeout shorter than function timeout
Messages are redelivered while still processing.
DLQ configured on the Lambda instead of the queue
For SQS sources, failed messages go by the queue's redrive policy, so the function DLQ does not apply.
Unbounded scaling into a database
Cap concurrency to what the database can serve.
Frequently Asked Questions
What is a partial batch response?
It lets your function report which messages in a batch failed, so Lambda deletes the successful ones and only the failed ones return to the queue.
How do I limit Lambda concurrency from SQS?
Set maximum concurrency on the event source mapping, or reserved concurrency on the function. The mapping setting limits the queue without taking from the function's other triggers.
Does a larger batch size always help?
No. It reduces invocations but makes each failure and retry costlier, and processing time per batch must fit the function timeout.
Want Help With Tuning Lambda SQS event source mappings?
Schedule an appointment. Tell us about your environment and we will outline the safest, highest-value approach.
Schedule Appointment › No commitment required