(aws-stepfunctions-tasks): (Batch SubmitJob automatically adds a containerOverrides even with an ECS multi container Batch )

[cbm-afoulfoin]

Describe the bug

Considering an AWS batch job definition that defines two containers within ecsProperties:

const jobDefinition = new CfnJobDefinition(scope, `job-definition-${name}`, {
        containerProperties: {
            ecsProperties: {
            ....
            }
       }
})

We try to launch that job throught AWS step function:

const batchJobTask = new tasks.BatchSubmitJob(scope, `${app}-${name}-job-task`, {
        jobName: `${app}-${name}-job`,
        ...
        stateName: 'ExecuteBatchJob',
      });

Step function will fails with the following error : Container overrides should not be set for ecsProperties jobs. (Service: AWSBatch; Status Code: 400; Error Code: ClientException; Request ID: 36d5ba3a-466d-4ed8-b559-7a3193c0970b; Proxy: null)

It seems that AWS Step function automatically adds this bloc in the batch definition:

"ContainerOverrides":{
    "Environment": [
      {
        "Name": "MANAGED_BY_AWS",
        "Value": "STARTED_BY_STEP_FUNCTIONS"
      }
    ]
  }

But ContainerOverrides seems incompatibale with a job that uses EcsProperties, making the Step failing.

Another example of user having the same issue: https://repost.aws/questions/QUYUtrKcPGRf-Rj1m1dXJH1Q/aws-batch-running-ecsproperties-job-with-aws-stepfunction

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Library Version

No response

Expected Behavior

The BatchSubmitJob should work with AWS Batch jobs that define ecsProperties

Current Behavior

The BatchSubmitJob doesn't work with AWS Batch jobs that define ecsProperties

Reproduction Steps

Possible Solution

No response

Additional Information/Context

No response

AWS CDK Library version (aws-cdk-lib)

2.212.0

AWS CDK CLI version

2.1020.0

Node.js Version

22

OS

Windows

Language

TypeScript

Language Version

No response

Other information

No response

[pahud]

Hi @cbm-afoulfoin,

Thank you for reporting this issue. I've successfully reproduced the problem and investigated the root cause.

Reproduction

I created a minimal CDK app with a Batch job definition using ecsProperties (multi-container) and a Step Functions state machine with BatchSubmitJob:

// Job Definition with ecsProperties (multi-container)
const jobDef = new batch.CfnJobDefinition(this, 'JobDef', {
  type: 'container',
  platformCapabilities: ['FARGATE'],
  ecsProperties: {
    taskProperties: [
      {
        executionRoleArn: executionRole.roleArn,
        taskRoleArn: executionRole.roleArn,
        containers: [
          {
            name: 'main-container',
            image: 'public.ecr.aws/amazonlinux/amazonlinux:latest',
            command: ['echo', 'hello'],
            resourceRequirements: [
              { type: 'VCPU', value: '0.25' },
              { type: 'MEMORY', value: '512' },
            ],
          },
          {
            name: 'sidecar-container',
            image: 'public.ecr.aws/amazonlinux/amazonlinux:latest',
            command: ['echo', 'sidecar'],
            resourceRequirements: [
              { type: 'VCPU', value: '0.25' },
              { type: 'MEMORY', value: '512' },
            ],
          },
        ],
      },
    ],
  },
});

// Step Functions task
const batchTask = new tasks.BatchSubmitJob(this, 'BatchTask', {
  jobDefinitionArn: jobDef.ref,
  jobName: 'test-job',
  jobQueueArn: jobQueue.jobQueueArn,
});

new sfn.StateMachine(this, 'StateMachine', {
  definition: batchTask,
});

CDK Synthesized Output

The CDK correctly synthesizes the state machine without ContainerOverrides:

{
  "Type": "Task",
  "Resource": "arn:aws:states:::batch:submitJob.sync",
  "Parameters": {
    "JobDefinition": "arn:aws:batch:...:job-definition/JobDef-...:1",
    "JobName": "test-job",
    "JobQueue": "arn:aws:batch:...:job-queue/JobQueue..."
  },
  "End": true
}

Note: No ContainerOverrides in the CDK-generated definition.

Execution Result

When the state machine executes, it fails with:

Batch.ClientException: Container overrides should not be set for ecsProperties jobs.
(Service: AWSBatch; Status Code: 400; Error Code: ClientException)

Examining the execution history reveals that AWS Step Functions service automatically injects ContainerOverrides at runtime:

{
  "parameters": {
    "JobDefinition": "arn:aws:batch:...",
    "JobName": "test-job",
    "JobQueue": "arn:aws:batch:...",
    "ContainerOverrides": {
      "Environment": [
        {
          "Name": "MANAGED_BY_AWS",
          "Value": "STARTED_BY_STEP_FUNCTIONS"
        }
      ]
    }
  }
}

Analysis

This appears to be a service-level limitation rather than a CDK bug. The AWS Step Functions service automatically adds ContainerOverrides with tracking metadata for job management, but according to the AWS Batch SubmitJob API, ContainerOverrides is only compatible with job definitions using containerProperties, not ecsProperties. For ecsProperties jobs, the API expects ecsTaskOverride instead.

The CDK construct is working as designed - it doesn't add any container overrides. The issue occurs at the AWS Step Functions service layer when it injects tracking metadata.

Next Steps

I'll reach out internally to the AWS Step Functions and Batch service teams for guidance on how to properly support ecsProperties jobs in Step Functions workflows. This may require service-level changes to detect ecsProperties jobs
and use ecsTaskOverride instead of ContainerOverrides.

I'll update this issue once I have more information from the service teams.