AWS ECS Support
ECS deployment circuit breaker and automatic rollback.
A rolling ECS deployment with a broken image can keep launching tasks that fail, and leave the service stuck for hours. The deployment circuit breaker detects repeated failures and can roll the service back to the last healthy version. We configure it with the settings that make it reliable.
When You Need This
The situations where this work pays for itself.
Signs it is time
- A bad release leaves the service flapping or stuck in a deployment
- Rollbacks are manual and happen at 2 a.m.
- Deployments hang in progress with tasks failing to start
- You want a failed health check to stop a release automatically
- You use CodePipeline or CI to update task definitions and need safe failure
What you get
- Service definitions with circuit breaker and capacity settings in code
- Alarm-based rollback tied to your key metrics
- EventBridge notifications for deployment failure and rollback
- A tested failure drill for staging
How we do it
A clear, safe process you can follow along with.
1. Check the deployment controller
The circuit breaker applies to services using the ECS rolling update deployment controller. Services using CodeDeploy blue/green roll back through CodeDeploy instead. We confirm which controller each service uses.
2. Enable the breaker with rollback
In the service deployment configuration we turn on deploymentCircuitBreaker with enable and rollback set to true. When tasks of the new deployment keep failing to reach a running and healthy state, the deployment is marked failed and the service returns to the last completed deployment.
3. Set the capacity settings deliberately
minimumHealthyPercent and maximumPercent decide how many old tasks stay up while new ones start. For production we normally keep minimumHealthyPercent at 100 with room above 100 for surge, so a failed deployment never reduces capacity.
4. Make health checks meaningful
The breaker counts tasks that fail to start or fail health checks. We make sure the load balancer health check path reflects real readiness, and set the health check grace period long enough for slow starts so healthy tasks are not counted as failures.
5. Add deployment alarms
We attach CloudWatch alarms, for example on target 5xx rate or latency, to the deployment so a release that starts but behaves badly also rolls back. This catches problems the task-level checks miss.
6. Notify and test the failure path
We send ECS deployment state change events from EventBridge to chat or paging, then deploy a deliberately broken image in staging to see the rollback happen end to end.
Mistakes we help you avoid
Common ways this goes wrong, and what we do instead.
Enabling the breaker without rollback
The deployment fails but the service stays on the broken version.
minimumHealthyPercent too low
A bad rollout can remove most healthy tasks before it fails.
Rolling back infrastructure but not data
If the release ran a migration, an old image may not work. Migrations must be backward compatible.
Health check that always returns 200
It lets broken versions pass. Readiness must test real dependencies carefully.
Frequently Asked Questions
What is the ECS deployment circuit breaker?
It is a rolling update feature that detects a deployment that cannot reach a steady state, marks it failed, and optionally rolls back to the last good deployment.
Does the circuit breaker work with CodeDeploy blue/green?
No. Blue/green services use CodeDeploy for traffic shifting and rollback, with its own alarms and hooks.
Why is my ECS deployment stuck in progress?
Usually new tasks fail to start or fail health checks, for example image pull errors, memory kills, wrong ports or a health check path that returns an error. The service events and stopped task reasons show which.
Want Help With ECS deployment circuit breaker and automatic rollback?
Schedule an appointment. Tell us about your environment and we will outline the safest, highest-value approach.
Schedule Appointment › No commitment required