Skip to content

Add Nextflow head job IAM role to manual AWS Batch setup docs#1219

Open
robsyme wants to merge 5 commits intomasterfrom
fix/ec2-headjob-iam-permissions
Open

Add Nextflow head job IAM role to manual AWS Batch setup docs#1219
robsyme wants to merge 5 commits intomasterfrom
fix/ec2-headjob-iam-permissions

Conversation

@robsyme
Copy link
Copy Markdown
Member

@robsyme robsyme commented Mar 22, 2026

Summary

  • Adds a new Create a Nextflow head job role section to the manual AWS Batch setup docs (enterprise and cloud)
  • Provides a complete IAM policy with explicit logs:GetLogEvents permission, which Nextflow requires to retrieve task stderr from CloudWatch when tasks fail
  • Adds the role to the numbered steps list and cross-references it from the head queue compute environment setup
  • Clarifies the distinction between the head job role (attached to the Nextflow container) and the EC2 instance role (applied to the underlying instance)

Background

When Batch Forge creates a Fargate head job, the platform automatically attaches the correct IAM policy including logs:GetLogEvents. For EC2-based manual compute environments, customers must create this role themselves — but the docs previously provided no guidance on what permissions it needs. Customers who created a least-privilege role without referencing logs:Get* wildcards would hit an AccessDeniedException when Nextflow tried to fetch task logs on failure, with a misleading HTTP 400 error.

Test plan

  • Verify both enterprise and cloud versions of manual-aws-batch-setup.mdx render correctly
  • Confirm the new section appears between the EC2 instance role and SpotFleet role sections
  • Confirm the head queue compute environment tab references seqera-headjob-role

🤖 Generated with Claude Code

EC2-based manual compute environments require a dedicated head job role
with explicit CloudWatch Logs permissions, including logs:GetLogEvents.
Without this permission, Nextflow cannot retrieve task stderr from
CloudWatch when tasks fail, producing an AccessDeniedException instead
of the actual error.

Batch Forge auto-creates this role for Fargate head jobs, but customers
using EC2 head jobs must create it manually. The docs previously had no
guidance on this role.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Signed-off-by: Rob Syme <rob.syme@gmail.com>
@netlify
Copy link
Copy Markdown

netlify bot commented Mar 22, 2026

Deploy Preview for seqera-docs ready!

Name Link
🔨 Latest commit 557f622
🔍 Latest deploy log https://app.netlify.com/projects/seqera-docs/deploys/69c6e1a960cd0f00078de9c4
😎 Deploy Preview https://deploy-preview-1219--seqera-docs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@justinegeffen justinegeffen added 1. Editor review Needs a language review 1. Dev/PM/SME Needs a review by a Dev/PM/SME labels Mar 23, 2026
@rnaidu-seqera
Copy link
Copy Markdown
Contributor

@robsyme do we want to include S3 category permissions in the docs too? I ask because I've seen a ticket or two recently where users want to supply their own custom head job role in Platform and have that head job role interact with a specific S3 bucket (for scratch etc)

@justinegeffen justinegeffen removed the 1. Editor review Needs a language review label Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1. Dev/PM/SME Needs a review by a Dev/PM/SME

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants