AWS · lesson 18 of 19
Beginner
4Intermediate
13- 5 VPC and Networking Basics
- 6 RDS Basics
- 7 AWS Lambda Fundamentals
- 8 CloudFormation Basics
- 9 CloudWatch Monitoring Basics
- 11 DynamoDB: Tables and Keys
- 12 DynamoDB: Scan, Reserved Words, and Global Secondary Indexes
- 13 DynamoDB: Conditional Writes, Atomic Counters, and Streams
- 14 SQS: Visibility Timeout and Dead-Letter Queues
- 15 SQS: FIFO Queues, Message Groups, and Deduplication
- 16 SNS: Fan-Out to SQS and Filter Policies
- 17 ELB and Auto Scaling: Wiring an ASG Behind a Load Balancer
- 18 ECS: Task Definitions and Services
Also part of AWS
8- + Linting AWS IAM Policies Before Deploy with Parliament from Cloud Security
- + Auditing the AWS IAM Credential Report with Python from Cloud Security
- + Hunting in AWS CloudTrail Logs with Python from Cloud Security
- + Dead-Letter Queues: What Happens When Retries Keep Failing from Event-Driven Architecture
- + Idempotent Consumers: Surviving a Duplicate Delivery from Event-Driven Architecture
- + Load Balancing: Round Robin vs Least Connections from System Design
- + Kubernetes Fundamentals from Kubernetes
- + Horizontal Pod Autoscaling: Real Replicas, Real CPU Numbers from Kubernetes
ECS: Task Definitions and Services
Register two revisions of an ECS task definition, create a service that reports desiredCount without ever running anything, then docker run the same image/port/cpu/memory/env to see what a task actually is — against moto.
By SkillByExample · Updated
An ECS task definition is a versioned specification, not a running thing; a service keeps
a desired number of tasks of one task definition running. This lesson registers two task
definition revisions and a service against moto — which tracks the numbers but runs nothing —
then runs the task definition’s own image, port, CPU/memory, and environment for real with
docker run, to show what a task specification actually describes.
Two Task Definition Revisions
#!/usr/bin/env bash
set -e
aws() { MSYS_NO_PATHCONV=1 docker run --rm --network aws18-lab -e AWS_ACCESS_KEY_ID=test -e AWS_SECRET_ACCESS_KEY=test -e AWS_DEFAULT_REGION=us-east-1 amazon/aws-cli:2.39.5 "$@" --endpoint-url http://aws18-moto:5000; }
aws ecs create-cluster --cluster-name demo-cluster --query 'cluster.{clusterName:clusterName,status:status}'
aws ecs register-task-definition --family web-task \
--container-definitions '[{"name":"web","image":"nginx:1.27-alpine","cpu":128,"memory":256,"portMappings":[{"containerPort":80,"hostPort":80}],"environment":[{"name":"ENV","value":"demo"}]}]' \
--query 'taskDefinition.{family:family,revision:revision,memory:containerDefinitions[0].memory}'
aws ecs register-task-definition --family web-task \
--container-definitions '[{"name":"web","image":"nginx:1.27-alpine","cpu":128,"memory":512,"portMappings":[{"containerPort":80,"hostPort":80}],"environment":[{"name":"ENV","value":"demo"}]}]' \
--query 'taskDefinition.{family:family,revision:revision,memory:containerDefinitions[0].memory}'
echo "all revisions:"
aws ecs list-task-definitions --family-prefix web-task --query 'taskDefinitionArns'
$ docker network create aws18-lab
bfa326cad136a9fa964755afdb17ce3062c3ff1fcc074764ebb39c9dc78446be
$ docker run -d --name aws18-moto --network aws18-lab motoserver/moto:5.2.5
d10d954549b7bced99b8f344f5ae3acb2881068adcfcaacb4cf98df68497b670
$ until docker run --rm --network aws18-lab curlimages/curl:8.11.1 -s -o /dev/null http://aws18-moto:5000; do sleep 0.3; done
$ bash setup.sh
{
"clusterName": "demo-cluster",
"status": "ACTIVE"
}
{
"family": "web-task",
"revision": 1,
"memory": 256
}
{
"family": "web-task",
"revision": 2,
"memory": 512
}
all revisions:
[
"arn:aws:ecs:us-east-1:123456789012:task-definition/web-task:1",
"arn:aws:ecs:us-east-1:123456789012:task-definition/web-task:2"
]
Two independent revisions under the same family, differing only in memory — register-task-definition
never checks that an older revision exists for comparison, it just adds the next revision number.
A Service That Reports a Number It Never Reaches
#!/usr/bin/env bash
set -e
aws() { MSYS_NO_PATHCONV=1 docker run --rm --network aws18-lab -e AWS_ACCESS_KEY_ID=test -e AWS_SECRET_ACCESS_KEY=test -e AWS_DEFAULT_REGION=us-east-1 amazon/aws-cli:2.39.5 "$@" --endpoint-url http://aws18-moto:5000; }
aws ecs create-service --cluster demo-cluster --service-name web-service \
--task-definition web-task:2 --desired-count 2 --launch-type EC2 \
--query 'service.{serviceName:serviceName,desiredCount:desiredCount,runningCount:runningCount,status:status}'
$ bash create_service.sh
{
"serviceName": "web-service",
"desiredCount": 2,
"runningCount": 0,
"status": "ACTIVE"
}
desiredCount: 2, runningCount: 0, status: ACTIVE — CreateService accepted the request and
is tracking the number, but moto has no scheduler: the real ECS component that would place
tasks onto capacity and move runningCount toward desiredCount doesn’t exist here. Waiting
longer changes nothing; there is no process anywhere that would ever update it. This particular
gap would show up on real AWS too, for a different reason: demo-cluster has no EC2 instances
registered as container-instance capacity, so a real scheduler would also have nowhere to place a
task — it would just say so, with an “unable to place a task” event, instead of staying silent.
What a Task Actually Is: Running the Same Image for Real
#!/usr/bin/env bash
set -e
docker run -d --name aws18-task --memory=512m --cpu-shares=128 -p 8080:80 -e ENV=demo nginx:1.27-alpine
until curl -s -o /dev/null http://localhost:8080; do sleep 0.3; done
echo "http status:"
curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8080
echo "env var inside the container:"
docker exec aws18-task printenv ENV
$ bash run_task.sh
bd8d05118818418b5d803a626e74a357df175bcc95ac028cdd8411f24211f6a7
http status:
200
env var inside the container:
demo
Every value here came straight from web-task:2’s own JSON: nginx:1.27-alpine is the
container’s image, 512m comes from memory: 512 (MiB, a hard cap either way), and --cpu-shares=128
comes from cpu: 128 — on the EC2 launch type, a task definition’s container-level cpu maps to
Docker’s --cpu-shares, a relative weight for CPU time under contention (Docker’s own default is
1024), not a hard fraction-of-a-vCPU limit; an idle host still lets this container use 100% of a
core if nothing else is competing for it. 8080 is this lesson’s own choice for the host side
(so it doesn’t collide with anything else on localhost) mapped to the task definition’s
containerPort: 80; the real hostPort: 80 from the JSON is swapped out here, not mirrored.
ENV=demo is the task definition’s own environment entry. This is what a real EC2 instance’s ECS
agent would have run on the service’s behalf — done by hand here because moto never will.
Practice
1. Register a Fargate-compatible task definition and create a Fargate service with a real subnet and security group, confirming it hits the same silent non-scheduling as EC2.
#!/usr/bin/env bash
set -e
aws() { MSYS_NO_PATHCONV=1 docker run --rm --network aws18-lab -e AWS_ACCESS_KEY_ID=test -e AWS_SECRET_ACCESS_KEY=test -e AWS_DEFAULT_REGION=us-east-1 amazon/aws-cli:2.39.5 "$@" --endpoint-url http://aws18-moto:5000; }
aws ecs register-task-definition --family web-task-fg \
--requires-compatibilities FARGATE --network-mode awsvpc --cpu 256 --memory 512 \
--container-definitions '[{"name":"web","image":"nginx:1.27-alpine","portMappings":[{"containerPort":80,"protocol":"tcp"}]}]' \
--query 'taskDefinition.{family:family,revision:revision}'
SUBNET=$(aws ec2 describe-subnets --query 'Subnets[0].SubnetId' --output text)
SG=$(aws ec2 describe-security-groups --query 'SecurityGroups[0].GroupId' --output text)
aws ecs create-service --cluster demo-cluster --service-name web-fargate \
--task-definition web-task-fg:1 --desired-count 1 --launch-type FARGATE \
--network-configuration "{\"awsvpcConfiguration\":{\"subnets\":[\"$SUBNET\"],\"securityGroups\":[\"$SG\"]}}" \
--query 'service.{serviceName:serviceName,desiredCount:desiredCount,runningCount:runningCount}'
$ bash practice1.sh
{
"family": "web-task-fg",
"revision": 1
}
{
"serviceName": "web-fargate",
"desiredCount": 1,
"runningCount": 0
}
A properly Fargate-compatible task definition (requiresCompatibilities FARGATE, networkMode awsvpc, task-level cpu/memory) with a complete awsvpcConfiguration creates the service
without error — desiredCount: 1, runningCount: 0, the identical gap EC2 showed above. Leaving
securityGroups out of the same request does crash this version of moto with an HTTP 500 (a real,
reproducible gotcha worth knowing about if you hit it) — but that’s moto choking on a missing
optional field, not a real difference between how Fargate and EC2 fail to schedule.
2. Describe the EC2-launch-type service again after a longer wait and confirm runningCount still never moves.
#!/usr/bin/env bash
set -e
aws() { MSYS_NO_PATHCONV=1 docker run --rm --network aws18-lab -e AWS_ACCESS_KEY_ID=test -e AWS_SECRET_ACCESS_KEY=test -e AWS_DEFAULT_REGION=us-east-1 amazon/aws-cli:2.39.5 "$@" --endpoint-url http://aws18-moto:5000; }
sleep 3
aws ecs describe-services --cluster demo-cluster --services web-service \
--query 'services[0].{desiredCount:desiredCount,runningCount:runningCount,pendingCount:pendingCount}'
$ bash practice2.sh
{
"desiredCount": 2,
"runningCount": 0,
"pendingCount": 2
}
runningCount is still 0, and pendingCount sits at 2 — exactly desiredCount - runningCount
— and stays there no matter how long you wait. On a real ECS service pendingCount is a moving
signal that briefly rises while tasks are being placed and falls back to 0 as they start running;
here it’s a static arithmetic difference, because there’s no placement process running in the
background on moto at all to ever resolve it.
3. Update the service to point at task definition revision 1 instead of 2, and confirm the service record changes with no effect on what's running.
#!/usr/bin/env bash
set -e
aws() { MSYS_NO_PATHCONV=1 docker run --rm --network aws18-lab -e AWS_ACCESS_KEY_ID=test -e AWS_SECRET_ACCESS_KEY=test -e AWS_DEFAULT_REGION=us-east-1 amazon/aws-cli:2.39.5 "$@" --endpoint-url http://aws18-moto:5000; }
aws ecs update-service --cluster demo-cluster --service web-service --task-definition web-task:1 \
--query 'service.{taskDefinition:taskDefinition,runningCount:runningCount}'
$ bash practice3.sh
{
"taskDefinition": "web-task:1",
"runningCount": 0
}
taskDefinition now ends in :1, confirming the control-plane record changed — runningCount
is still 0, the same nothing-happens-on-moto gap as every other write to this service.
4. Deregister task definition revision 1 and confirm it moves to INACTIVE rather than disappearing.
#!/usr/bin/env bash
set -e
aws() { MSYS_NO_PATHCONV=1 docker run --rm --network aws18-lab -e AWS_ACCESS_KEY_ID=test -e AWS_SECRET_ACCESS_KEY=test -e AWS_DEFAULT_REGION=us-east-1 amazon/aws-cli:2.39.5 "$@" --endpoint-url http://aws18-moto:5000; }
aws ecs deregister-task-definition --task-definition web-task:1 --query 'taskDefinition.status'
aws ecs describe-task-definition --task-definition web-task:1 --query 'taskDefinition.status'
$ bash practice4.sh
"INACTIVE"
"INACTIVE"
Both calls return INACTIVE, not an error — a deregistered revision can still be described (and
is still what Practice 3’s service record points at), it just can’t be used in a new
CreateService/UpdateService call going forward.
Cleanup
$ docker rm -f aws18-moto aws18-task
aws18-moto
aws18-task
$ docker network rm aws18-lab
aws18-lab
Next: designing a single DynamoDB table so every access pattern is one Query.