Overview and What You Will Learn
A Docker CI/CD pipeline builds an image, scans it for vulnerabilities, pushes it to a registry, and deploys it, with each stage acting as a gate. If the scan finds serious CVEs nothing is pushed, and if the push fails nothing is deployed. This lab builds that pipeline in GitHub Actions with Amazon ECR and ECS, using short-lived OIDC credentials instead of stored AWS keys.
In this lab you will:
- Build an image once with BuildKit caching and tag it with the git commit SHA
- Scan the image with Trivy before it can reach a registry
- Authenticate to AWS with OIDC, with no long-lived access keys
- Push to ECR only from the main branch
- Deploy a new ECS task definition behind a manual approval gate
- Add multi-platform builds, a Kubernetes variant, and a rollback plan
Why This Matters in Production
Teams that ship many times a day, like those at Shopify or Atlassian, cannot rely on someone remembering to scan or approve a release. The pipeline enforces it every time. Tagging images with the commit SHA also means any running container can be traced to the exact code that produced it, which is what you need at 2 AM during an incident.
Core Principles
A trustworthy pipeline never pushes before scanning and never deploys without a reviewable record. Pull requests build and scan only, while main pushes and deploys.
Four rules shape the design:
- Scan before push: A vulnerable image should never exist in your registry.
- Tag with the commit SHA: Never rely on
latestto identify what is deployed. - Use OIDC, not stored keys: GitHub requests short-lived AWS credentials per run.
- Gate production: Use a GitHub environment with required reviewers for the deploy job.
Detailed Step-by-Step Practical Lab
How to Authenticate GitHub Actions to AWS With OIDC
OIDC lets a workflow assume an AWS IAM role without any stored access keys. You create an IAM role whose trust policy accepts GitHub's identity provider and restricts it to your repository and branch.
aws iam create-open-id-connect-provider \ --url https://token.actions.githubusercontent.com \ --client-id-list sts.amazonaws.comThen create two roles: one that can push to ECR (for the build job) and one that can update ECS (for the deploy job). Restrict each trust policy to repo:devops-network/payment-api:ref:refs/heads/main so only main can assume them.
How to Build, Scan, and Push a Docker Image in GitHub Actions
This workflow builds once, loads the image into the runner, scans it, and pushes only from main. Pull requests stop after the scan and never receive AWS credentials.
# .github/workflows/docker-pipeline.ymlname: Docker CI/CD Pipeline on: push: branches: [main] pull_request: branches: [main] concurrency: group: docker-pipeline-${{ github.ref }} cancel-in-progress: true env: AWS_REGION: ap-south-1 ECR_REPOSITORY: payment-api IMAGE_TAG: ${{ github.sha }} jobs: build-scan-push: runs-on: ubuntu-latest permissions: contents: read id-token: write security-events: write steps: - name: Checkout uses: actions/checkout@v4 - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Build image uses: docker/build-push-action@v6 with: context: . push: false load: true tags: ${{ env.ECR_REPOSITORY }}:${{ env.IMAGE_TAG }} cache-from: type=gha cache-to: type=gha,mode=max # Pin third-party actions to a release tag or commit SHA, never @master - name: Scan image with Trivy uses: aquasecurity/trivy-action@0.28.0 with: image-ref: ${{ env.ECR_REPOSITORY }}:${{ env.IMAGE_TAG }} format: sarif output: trivy-results.sarif severity: CRITICAL,HIGH ignore-unfixed: true exit-code: "1" - name: Upload scan results to GitHub Security if: always() uses: github/codeql-action/upload-sarif@v3 with: sarif_file: trivy-results.sarif - name: Configure AWS credentials (OIDC) if: github.ref == 'refs/heads/main' uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: arn:aws:iam::111122223333:role/github-actions-ecr-push aws-region: ${{ env.AWS_REGION }} - name: Login to Amazon ECR if: github.ref == 'refs/heads/main' id: login-ecr uses: aws-actions/amazon-ecr-login@v2 - name: Push to ECR if: github.ref == 'refs/heads/main' run: | FULL_IMAGE=${{ steps.login-ecr.outputs.registry }}/${{ env.ECR_REPOSITORY }}:${{ env.IMAGE_TAG }} docker tag ${{ env.ECR_REPOSITORY }}:${{ env.IMAGE_TAG }} "$FULL_IMAGE" docker push "$FULL_IMAGE"RememberEnable tag immutability on the ECR repository so a pushed SHA tag can never be overwritten.
How to Deploy a New Image to Amazon ECS From GitHub Actions
Forcing a new deployment alone redeploys the same task definition, so with SHA tags you must register a new task definition that points to the new image. The ECS actions do this and wait for the service to become stable.
deploy: runs-on: ubuntu-latest needs: build-scan-push if: github.ref == 'refs/heads/main' environment: production permissions: contents: read id-token: write steps: - name: Checkout uses: actions/checkout@v4 - name: Configure AWS credentials (OIDC) uses: aws-actions/configure-aws-credentials@v4 with: role-to-assume: arn:aws:iam::111122223333:role/github-actions-ecs-deploy aws-region: ap-south-1 - name: Render task definition with the new image id: render uses: aws-actions/amazon-ecs-render-task-definition@v1 with: task-definition: .aws/task-definition.json container-name: payment-api image: 111122223333.dkr.ecr.ap-south-1.amazonaws.com/payment-api:${{ github.sha }} - name: Deploy to ECS uses: aws-actions/amazon-ecs-deploy-task-definition@v2 with: task-definition: ${{ steps.render.outputs.task-definition }} service: payment-api cluster: mumbai-prod-cluster wait-for-service-stability: trueThe environment: production line pauses this job until a required reviewer approves it in GitHub. Configure the reviewers under Settings, then Environments.
How to Deploy the Same Image to Kubernetes Instead
If you run Kubernetes, replace the ECS steps with a rolling image update that waits for the rollout to finish. The SHA tag makes the update explicit and auditable.
kubectl set image deployment/payment-api \ payment-api=111122223333.dkr.ecr.ap-south-1.amazonaws.com/payment-api:$GITHUB_SHA \ -n production # Fail the job if the rollout does not completekubectl rollout status deployment/payment-api -n production --timeout=300sHow to Build Multi-Platform Docker Images in CI
Multi-platform builds produce one image that runs on AMD64 and ARM64, such as AWS Graviton. Trivy scans the loaded single-platform image, so scan the platform you deploy, then publish the multi-arch manifest.
- name: Set up QEMU uses: docker/setup-qemu-action@v3 - name: Build and push multi-platform image if: github.ref == 'refs/heads/main' uses: docker/build-push-action@v6 with: context: . platforms: linux/amd64,linux/arm64 push: true tags: ${{ steps.login-ecr.outputs.registry }}/${{ env.ECR_REPOSITORY }}:${{ env.IMAGE_TAG }} cache-from: type=gha cache-to: type=gha,mode=maxHow to Roll Back a Bad Docker Deployment
A rollback redeploys the last known-good version, so keep old task definition revisions and image tags. Roll back by pointing the service at the previous revision rather than rebuilding anything.
# ECS: list recent revisions, then point the service at the previous oneaws ecs list-task-definitions \ --family-prefix payment-api \ --sort DESC \ --max-items 3 \ --region ap-south-1 aws ecs update-service \ --cluster mumbai-prod-cluster \ --service payment-api \ --task-definition payment-api:41 \ --region ap-south-1 # Kubernetes: Deployments keep previous ReplicaSets for this purposekubectl rollout undo deployment/payment-api -n productionProduction Best Practices & Common Pitfalls
- Use OIDC role assumption: Static AWS keys in GitHub Secrets are a rotation and leak risk.
- Scan before you push: Fail the build on CRITICAL and HIGH findings that have fixes available.
- Tag images with the git SHA: Add other tags only if they are not what production deploys.
- Pin every third-party action: Use release tags or commit SHAs, never
@master. - Give PRs no cloud credentials: Build and scan only, push and deploy from main.
- Require approval for production: Use GitHub environments with required reviewers.
- Build once: Reuse the scanned image and BuildKit cache instead of rebuilding in later stages.
- Use separate roles per stage: The push role and the deploy role should have different, minimal permissions.
TipUpload scan results as SARIF so findings appear in the repository's Security tab, where they are tracked over time instead of buried in job logs.
Common MistakeDeploying with
--force-new-deploymentwhile the task definition still references a mutable tag likelatest. You cannot tell which code is running, and a rollback is guesswork.
SecurityRestrict the OIDC trust policy to your exact repository and branch. A policy that trusts any repository in your organisation lets unrelated workflows assume your deploy role.
Quick Reference & Troubleshooting Commands
| Task | Command |
|---|---|
| Find the image a container runs | docker inspect name --format '{{.Config.Image}}' |
| Scan a local image | trivy image --severity CRITICAL,HIGH image:tag |
| Check ECS rollout | aws ecs describe-services --cluster mumbai-prod-cluster --services payment-api |
| Check Kubernetes rollout | kubectl rollout status deployment/payment-api -n production |
| Roll back Kubernetes | kubectl rollout undo deployment/payment-api -n production |
| Show build cache usage | docker buildx du |
| Symptom | Likely cause | Fix |
|---|---|---|
Not authorized to perform sts:AssumeRoleWithWebIdentity |
Trust policy mismatch | Check repo, branch, and id-token: write permission |
| Scan fails the pipeline | CRITICAL or HIGH CVE found | Update the base image or dependency |
| Deploy job never starts | Waiting for environment approval | Approve it or adjust reviewers |
| ECS runs the old image | Task definition not updated | Render a new task definition with the SHA tag |
| Slow builds | Cache not used | Enable cache-from and cache-to with type=gha |
