Skip to content

Conversation

@pasevin
Copy link
Collaborator

@pasevin pasevin commented Jan 5, 2026

Summary

Fix ECS deployments to actually deploy new Docker images instead of reusing old ones.

Problem

The staging deployment has been showing old CSS styling for hours despite multiple successful builds. Investigation revealed:

  1. The workflow only runs aws ecs update-service --force-new-deployment
  2. This tells ECS to restart tasks using the current task definition
  3. The task definition has a pinned image tag (old commit SHA), not :latest
  4. So ECS keeps pulling the same old image on every deployment

Solution

Update the deploy job to:

  1. Get the current task definition from the ECS service
  2. Update it with the new image (using the commit SHA tag)
  3. Register a new task definition revision
  4. Deploy the service with the new task definition

This ensures each deployment uses the freshly built Docker image from that specific commit.

Changes

  • Updated docker-stg.yaml deploy job with proper task definition update flow
  • Updated docker-prod.yaml with the same fix

Testing

After merging, trigger the staging workflow to verify the new deployment properly updates the Docker image.

The previous deployment workflow only called force-new-deployment which
restarts tasks using the current task definition. If the task definition
has a pinned image tag, ECS would pull the old image.

This fix:
1. Gets the current task definition from the ECS service
2. Updates it with the new image (using commit SHA tag)
3. Registers a new task definition revision
4. Deploys the service with the new task definition

This ensures each deployment uses the freshly built Docker image.
@pasevin pasevin requested a review from a team as a code owner January 5, 2026 19:54
@pasevin pasevin requested a review from Copilot January 5, 2026 19:55
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a critical deployment issue where ECS deployments were reusing old Docker images instead of deploying newly built ones. The root cause was that update-service --force-new-deployment only restarts tasks using the current task definition, which had a pinned image tag. The fix updates the task definition with the new image SHA before deploying.

  • Updates the deployment flow to retrieve the current task definition, update it with the new commit-tagged image, register a new revision, and then deploy
  • Applies the same fix to both staging and production workflows
  • Standardizes quote styles throughout the workflow files from single to double quotes

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 9 comments.

File Description
.github/workflows/docker-stg.yaml Replaces simple force-new-deployment with full task definition update flow for staging; standardizes quotes
.github/workflows/docker-prod.yaml Applies the same task definition update logic to production deployments

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Fix integer comparison: remove quotes around 0 in -eq comparison
- Add TASK_DEF_ARN validation after describe-services call
- Add error handling for describe-task-definition command
- Add JSON validation for task definition output
- Add NEW_TASK_DEF_ARN validation after register-task-definition
- Consistent whitespace formatting across both workflow files
@pasevin pasevin merged commit fafb950 into main Jan 5, 2026
11 checks passed
@pasevin pasevin deleted the fix/ecs-deploy-task-definition branch January 5, 2026 20:18
Comment on lines +267 to +268
NEW_IMAGE="${REGISTRY}/contracts-ui-builder-stg:${{ github.sha }}"
echo "Target image: $NEW_IMAGE"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be tagged always with github.sha and also latest...for staging atleast

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both tags are already pushed in the build step (lines 189-190 and 222-223). The deploy intentionally uses the SHA tag for deterministic, traceable deployments

pasevin added a commit that referenced this pull request Jan 5, 2026
Reverts the complex task definition update logic from PR #294.
The simple --force-new-deployment is sufficient since the task
definition uses :latest tag.
pasevin added a commit that referenced this pull request Jan 5, 2026
…295)

* revert(ci): restore simple ECS force-new-deployment

Reverts the complex task definition update logic from PR #294.
The simple --force-new-deployment is sufficient since the task
definition uses :latest tag.

* fix(builder): scan both node_modules locations for Tailwind classes

In Docker builds, .npmrc is excluded via .dockerignore, so pnpm's
shamefully-hoist=true doesn't apply. Packages install in
packages/builder/node_modules/ instead of root.

Now scans both locations:
- Root node_modules (local dev with shameful hoisting)
- Builder's node_modules (Docker builds without .npmrc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants