ci: Enable AWS runners by chtruong814 · Pull Request #1388 · NVIDIA-NeMo/Curator

chtruong814 · 2026-01-16T23:47:26Z

Description

ci: Enable AWS runners

The install test had to be written a little differently because running the GHA job directly in the container does not work yet on our self-hosted runners. So I had to rewrite to manually start the container and install from there.

Usage

# Add snippet demonstrating usage

Checklist

I am familiar with the Contributing Guide.
New or Existing tests cover these changes.
The documentation is up to date with these changes.

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps

Additional Comments (1)

.github/actions/test-template/action.yml, line 90 (link)

logic: references undefined step output steps.azure-fileshare.outputs.mnt_path - the azure-fileshare step was removed in this PR, so this variable will be empty and may cause issues

_{3 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-01-17T00:03:13Z

@@ -427,9 +409,9 @@ jobs:
  all-install-import-test:
    needs: [pre-flight]


logic: missing cuda-pre-flight in needs array - line 414 references needs.cuda-pre-flight.outputs.cuda_base_image but the job doesn't declare it as a dependency

Suggested change

needs: [pre-flight]

needs: [pre-flight, cuda-pre-flight]

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps

Additional Comments (1)

.github/actions/test-template/action.yml, line 100 (link)

logic: references deleted step output steps.azure-fileshare.outputs.mnt_path - this step was removed but the variable assignment remains

_{3 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps

Additional Comments (1)

.github/actions/test-template/action.yml, line 100 (link)

style: Dead code - references removed azure-fileshare step and MNT_PATH is never used

_{3 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps

Additional Comments (1)

.github/actions/test-template/action.yml, line 100 (link)

style: unused variable assignment referencing removed step

_{3 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps

Additional Comments (1)

.github/actions/test-template/action.yml, line 100 (link)

style: unused variable references removed step

MNT_PATH references steps.azure-fileshare.outputs.mnt_path but the azure-fileshare step was removed. This line should be deleted.

_{3 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps

_{3 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-01-23T01:52:18Z

+    - name: Install uuidgen
+      shell: bash -x -e -u -o pipefail {0}
+      if: ${{ contains(inputs.runner, 'aws') }}
      run: |
-        sudo apt update
-        sudo apt install -y cifs-utils
-
-        RESOURCE_GROUP_NAME="azure-gpu-vm-runner_group"
-        STORAGE_ACCOUNT_NAME="nemocistorageaccount2"
-        FILE_SHARE_NAME="fileshare"
-
-        MNT_ROOT="/media"
-        MNT_PATH="$MNT_ROOT/$STORAGE_ACCOUNT_NAME/$FILE_SHARE_NAME"
-
-        echo "MNT_PATH=$MNT_PATH" | tee -a "$GITHUB_OUTPUT"
-
-        sudo mkdir -p $MNT_PATH
-
-        # Create a folder to store the credentials for this storage account and
-        # any other that you might set up.
-        CREDENTIAL_ROOT="/etc/smbcredentials"
-        sudo mkdir -p "/etc/smbcredentials"
-
-        # Get the storage account key for the indicated storage account.
-        # You must be logged in with az login and your user identity must have
-        # permissions to list the storage account keys for this command to work.
-        STORAGE_ACCOUNT_KEY=$(az storage account keys list \
-            --resource-group $RESOURCE_GROUP_NAME \
-            --account-name $STORAGE_ACCOUNT_NAME \
-            --query "[0].value" --output tsv | tr -d '"')
-
-        # Create the credential file for this individual storage account
-        SMB_CREDENTIAL_FILE="$CREDENTIAL_ROOT/$STORAGE_ACCOUNT_NAME.cred"
-        if [ ! -f $SMB_CREDENTIAL_FILE ]; then
-            echo "username=$STORAGE_ACCOUNT_NAME" | sudo tee $SMB_CREDENTIAL_FILE > /dev/null
-            echo "password=$STORAGE_ACCOUNT_KEY" | sudo tee -a $SMB_CREDENTIAL_FILE > /dev/null
-        else
-            echo "The credential file $SMB_CREDENTIAL_FILE already exists, and was not modified."
-        fi
-
-        # Change permissions on the credential file so only root can read or modify the password file.
-        sudo chmod 600 $SMB_CREDENTIAL_FILE
-
-        # This command assumes you have logged in with az login
-        HTTP_ENDPOINT=$(az storage account show --resource-group $RESOURCE_GROUP_NAME --name $STORAGE_ACCOUNT_NAME --query "primaryEndpoints.file" --output tsv | tr -d '"')
-        SMB_PATH=$(echo $HTTP_ENDPOINT | cut -c7-${#HTTP_ENDPOINT})$FILE_SHARE_NAME
-
-        STORAGE_ACCOUNT_KEY=$(az storage account keys list --resource-group $RESOURCE_GROUP_NAME --account-name $STORAGE_ACCOUNT_NAME --query "[0].value" --output tsv | tr -d '"')
-
-        sudo mount -t cifs $SMB_PATH $MNT_PATH -o credentials=$SMB_CREDENTIAL_FILE,serverino,nosharesock,actimeo=30,mfsymlinks
-
-        ls -al $MNT_PATH/TestData
+        apt-get update
+        apt-get install -y uuid-runtime


running apt-get without sudo while shell doesn't specify root privileges - this may fail on non-root runners

Suggested change

- name: Install uuidgen

shell: bash -x -e -u -o pipefail {0}

if: ${{ contains(inputs.runner, 'aws') }}

run: |

sudo apt update

sudo apt install -y cifs-utils

RESOURCE_GROUP_NAME="azure-gpu-vm-runner_group"

STORAGE_ACCOUNT_NAME="nemocistorageaccount2"

FILE_SHARE_NAME="fileshare"

MNT_ROOT="/media"

MNT_PATH="$MNT_ROOT/$STORAGE_ACCOUNT_NAME/$FILE_SHARE_NAME"

echo "MNT_PATH=$MNT_PATH" | tee -a "$GITHUB_OUTPUT"

sudo mkdir -p $MNT_PATH

# Create a folder to store the credentials for this storage account and

# any other that you might set up.

CREDENTIAL_ROOT="/etc/smbcredentials"

sudo mkdir -p "/etc/smbcredentials"

# Get the storage account key for the indicated storage account.

# You must be logged in with az login and your user identity must have

# permissions to list the storage account keys for this command to work.

STORAGE_ACCOUNT_KEY=$(az storage account keys list \

--resource-group $RESOURCE_GROUP_NAME \

--account-name $STORAGE_ACCOUNT_NAME \

--query "[0].value" --output tsv | tr -d '"')

# Create the credential file for this individual storage account

SMB_CREDENTIAL_FILE="$CREDENTIAL_ROOT/$STORAGE_ACCOUNT_NAME.cred"

if [ ! -f $SMB_CREDENTIAL_FILE ]; then

echo "username=$STORAGE_ACCOUNT_NAME" | sudo tee $SMB_CREDENTIAL_FILE > /dev/null

echo "password=$STORAGE_ACCOUNT_KEY" | sudo tee -a $SMB_CREDENTIAL_FILE > /dev/null

else

echo "The credential file $SMB_CREDENTIAL_FILE already exists, and was not modified."

fi

# Change permissions on the credential file so only root can read or modify the password file.

sudo chmod 600 $SMB_CREDENTIAL_FILE

# This command assumes you have logged in with az login

HTTP_ENDPOINT=$(az storage account show --resource-group $RESOURCE_GROUP_NAME --name $STORAGE_ACCOUNT_NAME --query "primaryEndpoints.file" --output tsv | tr -d '"')

SMB_PATH=$(echo $HTTP_ENDPOINT | cut -c7-${#HTTP_ENDPOINT})$FILE_SHARE_NAME

STORAGE_ACCOUNT_KEY=$(az storage account keys list --resource-group $RESOURCE_GROUP_NAME --account-name $STORAGE_ACCOUNT_NAME --query "[0].value" --output tsv | tr -d '"')

sudo mount -t cifs $SMB_PATH $MNT_PATH -o credentials=$SMB_CREDENTIAL_FILE,serverino,nosharesock,actimeo=30,mfsymlinks

ls -al $MNT_PATH/TestData

apt-get update

apt-get install -y uuid-runtime

- name: Install uuidgen

shell: bash -x -e -u -o pipefail {0}

if: ${{ contains(inputs.runner, 'aws') }}

run: |

sudo apt-get update

sudo apt-get install -y uuid-runtime

greptile-apps · 2026-01-23T01:52:22Z

+          cat > ${{ github.workspace }}/.run_install.sh << 'SCRIPT'
+          set -e
+          apt-get update && apt-get install -y --no-install-recommends git curl
+          curl -LsSf https://astral.sh/uv/install.sh | sh


piping remote script directly to shell (curl | sh) creates security risk - if connection is interrupted or URL is compromised, arbitrary code could execute

greptile-apps · 2026-01-23T01:52:24Z

Additional Comments (1)

.github/actions/test-template/action.yml
references undefined variable - steps.azure-fileshare.outputs.mnt_path is no longer available after Azure Fileshare step was removed

        # MNT_PATH is no longer needed as we use the parameterized test-data-path

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps

_{4 files reviewed, 4 comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-01-23T02:00:04Z

+        apt-get update
+        apt-get install -y uuid-runtime


apt-get commands run without sudo but shell doesn't specify root context - may fail on runners without root privileges

greptile-apps · 2026-01-23T02:00:06Z

+          uv run python -c "import sys; print('Python:', sys.version);"
+          uv run python -c "from nemo_curator import package_info; print(f'nemo_curator v{package_info.__version__}')"
+          uv run python -c "import cudf, cuml; print('cudf, cuml');"
+          uv run python -c "import torch; print(f'torch {torch.__version__} (CUDA: {assert not torch.cuda.is_available()})');"


invalid Python syntax - assert is a statement, not an expression; this will cause a SyntaxError

Suggested change

uv run python -c "import torch; print(f'torch {torch.__version__} (CUDA: {assert not torch.cuda.is_available()})');"

uv run python -c "import torch; print(f'torch {torch.__version__} (CUDA: {torch.cuda.is_available()})');"

greptile-apps · 2026-01-23T02:00:08Z

+          cat > ${{ github.workspace }}/.run_install.sh << 'SCRIPT'
+          set -e
+          apt-get update && apt-get install -y --no-install-recommends git curl
+          curl -LsSf https://astral.sh/uv/install.sh | sh


piping remote script to shell (curl | sh) without verification creates security risk - if connection is interrupted or compromised, arbitrary code could execute

greptile-apps · 2026-01-23T02:00:10Z

Additional Comments (1)

.github/actions/test-template/action.yml
referencing removed step output - the azure-fileshare step was removed but this line still tries to access its output, will fail at runtime

        MNT_PATH=""

This reverts commit 09fdc5f. Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps

_{No files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps

_{No files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

ci: Enable AWS runners

e31a13f

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

chtruong814 requested a review from thomasdhc January 16, 2026 23:47

copy-pr-bot Bot temporarily deployed to test January 16, 2026 23:47 Inactive

copy-pr-bot Bot had a problem deploying to nemo-ci January 16, 2026 23:47 Error

copy-pr-bot Bot had a problem deploying to nemo-ci January 17, 2026 00:00 Failure

copy-pr-bot Bot temporarily deployed to nemo-ci January 17, 2026 00:00 Inactive

greptile-apps Bot reviewed Jan 17, 2026

View reviewed changes

Fix install test

8cb0cc5

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps Bot reviewed Jan 17, 2026

View reviewed changes

Restructure install test

a44ccb5

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps Bot reviewed Jan 17, 2026

View reviewed changes

chtruong814 added 2 commits January 17, 2026 02:34

Fix install test

483c050

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Fix install check

049802c

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps Bot reviewed Jan 17, 2026

View reviewed changes

Trying install test fix

224a41a

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps Bot reviewed Jan 17, 2026

View reviewed changes

chtruong814 added 2 commits January 23, 2026 01:35

Merge remote-tracking branch 'origin/main' into chtruong/migrate-aws

d553132

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

Fix install error

5dc39ce

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps Bot reviewed Jan 23, 2026

View reviewed changes

Test install failure

09fdc5f

Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps Bot reviewed Jan 23, 2026

View reviewed changes

Revert "Test install failure"

7276503

This reverts commit 09fdc5f. Signed-off-by: Charlie Truong <chtruong@nvidia.com>

greptile-apps Bot reviewed Jan 23, 2026

View reviewed changes

thomasdhc approved these changes Jan 27, 2026

View reviewed changes

Merge branch 'main' into chtruong/migrate-aws

245603c

greptile-apps Bot reviewed Jan 27, 2026

View reviewed changes

		@@ -427,9 +409,9 @@ jobs:
		all-install-import-test:
		needs: [pre-flight]

	uv run python -c "import torch; print(f'torch {torch.__version__} (CUDA: {assert not torch.cuda.is_available()})');"
	uv run python -c "import torch; print(f'torch {torch.__version__} (CUDA: {torch.cuda.is_available()})');"

Conversation

chtruong814 commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Usage

Checklist

Uh oh!

greptile-apps Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Additional Comments (1)

Uh oh!

greptile-apps Bot Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Additional Comments (1)

Uh oh!

greptile-apps Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Additional Comments (1)

Uh oh!

greptile-apps Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Additional Comments (1)

Uh oh!

greptile-apps Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Additional Comments (1)

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot commented Jan 23, 2026

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Jan 23, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot commented Jan 23, 2026

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

chtruong814 commented Jan 16, 2026 •

edited

Loading

greptile-apps Bot left a comment •

edited

Loading

greptile-apps Bot left a comment •

edited

Loading

greptile-apps Bot left a comment •

edited

Loading

greptile-apps Bot left a comment •

edited

Loading

greptile-apps Bot left a comment •

edited

Loading