AWS ECS Support
ECS task stopped with exit code 137: find out why and fix it.
Exit code 137 means the container received SIGKILL. In ECS that is most often an out-of-memory kill, but it can also come from a failed health check or a container that ignored SIGTERM. The stopped reason and metrics tell us which, and the fix differs for each.
When You Need This
The situations where this work pays for itself.
Signs it is time
- Tasks keep restarting and the service never gets to steady state
- The stopped reason says OutOfMemoryError: Container killed due to memory usage
- Tasks die under load or after running for some hours
- Deployments hang because new tasks are killed before passing health checks
- Java or Node.js containers use more memory than the limit you set
What you get
- Cause of the stops, backed by stoppedReason and metrics
- Corrected task definition with memory and health check settings
- Container shutdown handling fix
- CloudWatch alarms for memory and task churn
How we do it
A clear, safe process you can follow along with.
1. Read the stopped task details
We run aws ecs describe-tasks on recent stopped tasks, or use the console Stopped tab, and read stoppedReason, each container's exitCode and reason. A reason of OutOfMemoryError: Container killed due to memory usage settles it. Tasks stopped for failing the ELB health check say so in the reason.
2. Separate the possible causes
Exit 137 can be an OOM kill, an ECS stop after the stopTimeout elapsed with the app ignoring SIGTERM, or a health check stop. Exit 139 is a segmentation fault and exit 1 is an application error, which need different work.
3. Check memory settings against use
We compare the container memory hard limit and the task-level memory with MemoryUtilized in Container Insights. On Fargate the task size is what counts. On EC2 launch type, memoryReservation is only a soft limit while memory is a hard limit that triggers a kill.
4. Fix memory problems
We raise task memory to match the real peak plus margin, and make runtimes container aware, for example JVM heap flags such as MaxRAMPercentage and Node.js max-old-space-size, so the process stays below the limit. If memory climbs forever, we profile for a leak.
5. Fix health check kills
If the ALB or container health check stops the task, we check the health check path, the service health check grace period for slow starts, and security group access to the container port, since a container that is slow to boot is killed before it is ready.
6. Handle shutdown signals properly
We make sure the process is PID 1 or uses an init (initProcessEnabled), handles SIGTERM, and finishes within stopTimeout, so rolling deployments do not end in SIGKILL.
7. Verify
We deploy, watch stopped task counts and memory in Container Insights through a peak, and add alarms on memory utilization and on service task restarts.
Mistakes we help you avoid
Common ways this goes wrong, and what we do instead.
Raising memory without finding a leak
A leak just takes longer to hit the new limit.
Confusing soft and hard limits
memoryReservation does not stop a kill. memory does.
Health check grace period too short
The task is killed while still starting, then replaced, in a loop.
Shell wrapper as PID 1
Signals do not reach the app, so it gets SIGKILL after stopTimeout.
Frequently Asked Questions
What does exit code 137 mean in ECS?
The container was killed with SIGKILL, signal 9. In ECS that usually means it exceeded its memory limit, but a forced stop after the stop timeout also produces it.
How do I see why an ECS task stopped?
Use describe-tasks on the stopped task, or the Stopped tab of the cluster in the console, and read stoppedReason and the container reason. Stopped tasks are only kept for a limited time, so check promptly.
How much memory should an ECS task have?
Enough for observed peak usage plus headroom. Container Insights MemoryUtilized over a busy period is the right input, not a guess.
Want Help With ECS task stopped with exit code 137: find out why and fix it?
Schedule an appointment. Tell us about your environment and we will outline the safest, highest-value approach.
Schedule Appointment › No commitment required