-
Notifications
You must be signed in to change notification settings - Fork 238
Description
The cx attempts to resolve the issue by switching to different agent pools were unsuccessful, indicating the problem is not related to the agent environment.
A workaround was implemented by renaming the multiplier from 'OrcasBreadth.DirectorV2' to 'OrcasBreadthDirectorV2', which allowed the pipeline to function correctly.
2025-12-18T19:00:26.3299858Z ##[debug]Evaluating condition for step: 'Azure Key Vault: merucdtestinraakv'
2025-12-18T19:00:26.3304701Z ##[debug]Evaluating: and(succeeded(), notIn('', '1', 'true'))
2025-12-18T19:00:26.3305058Z ##[debug]Evaluating and:
2025-12-18T19:00:26.3307488Z ##[debug]..Evaluating succeeded:
2025-12-18T19:00:26.3308494Z ##[debug]..=> True
2025-12-18T19:00:26.3308770Z ##[debug]..Evaluating notIn:
2025-12-18T19:00:26.3310766Z ##[debug]....Evaluating String:
2025-12-18T19:00:26.3311460Z ##[debug]....=> ''
2025-12-18T19:00:26.3311885Z ##[debug]....Evaluating String:
2025-12-18T19:00:26.3312143Z ##[debug]....=> '1'
2025-12-18T19:00:26.3317232Z ##[debug]....Evaluating String:
2025-12-18T19:00:26.3317548Z ##[debug]....=> 'true'
2025-12-18T19:00:26.3317794Z ##[debug]..=> True
2025-12-18T19:00:26.3318323Z ##[debug]=> True
2025-12-18T19:00:26.3319692Z ##[debug]Expanded: and(True, notIn('', '1', 'true'))
2025-12-18T19:00:26.3319997Z ##[debug]Result: True
2025-12-18T19:00:26.3321389Z ##[section]Starting: Azure Key Vault: merucdtestinraakv
2025-12-18T19:00:26.3332669Z ==============================================================================
2025-12-18T19:00:26.3332824Z Task : Azure Key Vault
2025-12-18T19:00:26.3332927Z Description : Download Azure Key Vault secrets
2025-12-18T19:00:26.3333018Z Version : 2.264.0
2025-12-18T19:00:26.3333114Z Author : Microsoft Corporation
2025-12-18T19:00:26.3333223Z Help : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-key-vault
2025-12-18T19:00:26.3333339Z ==============================================================================
2025-12-18T19:00:26.3467722Z ##[debug]Resource Utilization warnings is disabled, switch "DISABLE_RESOURCE_UTILIZATION_WARNINGS" variable to "false" if you want to enable it
2025-12-18T19:00:26.3698286Z ##[debug]Using node path: C:\vss-agent\4.266.2\externals\node20_1\bin\node.exe
2025-12-18T19:00:26.8563651Z ##[debug]agent.TempDirectory=D:\a_work_temp
2025-12-18T19:00:26.8572530Z ##[debug]loading inputs and endpoints
2025-12-18T19:00:26.8590870Z ##[debug]loading ENDPOINT_AUTH_07fbf840-41d8-4a46-a17e-2eac8983aaa4
2025-12-18T19:00:26.8758667Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_07fbf840-41d8-4a46-a17e-2eac8983aaa4_SCOPE
2025-12-18T19:00:26.8761759Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_07fbf840-41d8-4a46-a17e-2eac8983aaa4_SERVICEPRINCIPALID
2025-12-18T19:00:26.8764819Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_07fbf840-41d8-4a46-a17e-2eac8983aaa4_TENANTID
2025-12-18T19:00:26.8767533Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_07fbf840-41d8-4a46-a17e-2eac8983aaa4_WORKLOADIDENTITYFEDERATIONISSUER
2025-12-18T19:00:26.8770391Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_07fbf840-41d8-4a46-a17e-2eac8983aaa4_WORKLOADIDENTITYFEDERATIONISSUERTYPE
2025-12-18T19:00:26.8773353Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_07fbf840-41d8-4a46-a17e-2eac8983aaa4_WORKLOADIDENTITYFEDERATIONSUBJECT
2025-12-18T19:00:26.8778088Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2025-12-18T19:00:26.8781398Z ##[debug]loading ENDPOINT_AUTH_SCHEME_07fbf840-41d8-4a46-a17e-2eac8983aaa4
2025-12-18T19:00:26.8786311Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2025-12-18T19:00:26.8789688Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2025-12-18T19:00:26.8803040Z ##[debug]loading INPUT_CONNECTEDSERVICENAME
2025-12-18T19:00:26.8806888Z ##[debug]loading INPUT_KEYVAULTNAME
2025-12-18T19:00:26.8811182Z ##[debug]loading INPUT_RUNASPREJOB
2025-12-18T19:00:26.8814458Z ##[debug]loading INPUT_SECRETSFILTER
2025-12-18T19:00:26.8848781Z ##[debug]loading SECRET_GETLATESTBUILDSFUNCTIONKEY
2025-12-18T19:00:26.8853903Z ##[debug]loading SECRET_ORCASDEPLOYMENTGATESFUNCTIONKEY
2025-12-18T19:00:26.8858709Z ##[debug]loading SECRET_ORCASDEVDEPLOYMENTGATESFUNCTIONKEY
2025-12-18T19:00:26.8862706Z ##[debug]loading SECRET_SECRETSTOREPASSWORD
2025-12-18T19:00:26.8866584Z ##[debug]loading SECRET_SECRETSTORESECRETS
2025-12-18T19:00:26.8871649Z ##[debug]loading SECRET_TARGETDEPLOYMENTGATESFUNCTIONKEY
2025-12-18T19:00:26.8876254Z ##[debug]loading SECRET_VMSS_ADMIN_PASSWORD
2025-12-18T19:00:26.8913876Z ##[debug]loaded 22
2025-12-18T19:00:26.8930707Z ##[debug]Agent.ProxyUrl=undefined
2025-12-18T19:00:26.8931356Z ##[debug]Agent.CAInfo=undefined
2025-12-18T19:00:26.8931969Z ##[debug]Agent.ClientCert=undefined
2025-12-18T19:00:26.8932530Z ##[debug]Agent.SkipCertValidation=undefined
2025-12-18T19:00:27.0972464Z ##[debug]agent.proxyurl=undefined
2025-12-18T19:00:27.0973135Z ##[debug]VSTS_ARM_REST_IGNORE_SSL_ERRORS=undefined
2025-12-18T19:00:27.0973662Z ##[debug]AZURE_HTTP_USER_AGENT=VSTS_8b119ea1-2e2a-4839-8db7-8c9e8d50f6fa_Release__1268017_3572677_4
2025-12-18T19:00:27.0974225Z ##[debug]check path : D:\a_work_tasks\AzureKeyVault_1e244d32-2dd4-4165-96fb-b7441ca9331e\2.264.0\node_modules\azure-pipelines-tasks-azure-arm-rest\module.json
2025-12-18T19:00:27.0976695Z ##[debug]adding resource file: D:\a_work_tasks\AzureKeyVault_1e244d32-2dd4-4165-96fb-b7441ca9331e\2.264.0\node_modules\azure-pipelines-tasks-azure-arm-rest\module.json
2025-12-18T19:00:27.0977261Z ##[debug]system.culture=en-US
2025-12-18T19:00:27.2106005Z ##[debug]Feature 'EnableMsalV3' = 'True'. Processed as 'true'.
2025-12-18T19:00:27.2400583Z ##[debug]Using msalv3
2025-12-18T19:00:27.2401205Z ##[debug]Resource file has already set to: D:\a_work_tasks\AzureKeyVault_1e244d32-2dd4-4165-96fb-b7441ca9331e\2.264.0\node_modules\azure-pipelines-tasks-azure-arm-rest\module.json
2025-12-18T19:00:27.2401827Z ##[debug]Agent.TempDirectory=D:\a_work_temp
2025-12-18T19:00:27.2402347Z ##[debug]Resource file has already set to: D:\a_work_tasks\AzureKeyVault_1e244d32-2dd4-4165-96fb-b7441ca9331e\2.264.0\node_modules\azure-pipelines-tasks-azure-arm-rest\module.json
2025-12-18T19:00:27.2424581Z ##[debug]Resource file has already set to: D:\a_work_tasks\AzureKeyVault_1e244d32-2dd4-4165-96fb-b7441ca9331e\2.264.0\node_modules\azure-pipelines-tasks-azure-arm-rest\module.json
2025-12-18T19:00:27.2427209Z ##[debug]Resource file has already set to: D:\a_work_tasks\AzureKeyVault_1e244d32-2dd4-4165-96fb-b7441ca9331e\2.264.0\node_modules\azure-pipelines-tasks-azure-arm-rest\module.json
2025-12-18T19:00:27.2438289Z ##[debug]RunAsPreJob=true
2025-12-18T19:00:27.2439837Z ##[debug]Setting resource path to D:\a_work_tasks\AzureKeyVault_1e244d32-2dd4-4165-96fb-b7441ca9331e\2.264.0\task.json
2025-12-18T19:00:27.2440625Z ##[debug]check path : D:\a_work_tasks\AzureKeyVault_1e244d32-2dd4-4165-96fb-b7441ca9331e\2.264.0\task.json
2025-12-18T19:00:27.2441650Z ##[debug]adding resource file: D:\a_work_tasks\AzureKeyVault_1e244d32-2dd4-4165-96fb-b7441ca9331e\2.264.0\task.json
2025-12-18T19:00:27.2442306Z ##[debug]system.culture=en-US
2025-12-18T19:00:27.2455254Z ##[debug]ConnectedServiceName=07fbf840-41d8-4a46-a17e-2eac8983aaa4
2025-12-18T19:00:27.2455999Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data SubscriptionId = b67c24d2-028d-4610-8370-2d0b793bec40
2025-12-18T19:00:27.2458017Z ##[debug]KeyVaultName=merucdtestinraakv
2025-12-18T19:00:27.2460169Z ##[debug]SecretsFilter=*
2025-12-18T19:00:27.2460833Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data AzureKeyVaultDnsSuffix = vault.azure.net
2025-12-18T19:00:27.2463222Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 auth param serviceprincipalid = ***
2025-12-18T19:00:27.2465191Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 auth scheme = WorkloadIdentityFederation
2025-12-18T19:00:27.2471306Z ##[debug]USE_MSAL=true
2025-12-18T19:00:27.2471747Z ##[debug]MSAL - USE_MSAL override is found: true
2025-12-18T19:00:27.2474246Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 auth scheme = WorkloadIdentityFederation
2025-12-18T19:00:27.2474744Z ##[debug]Overriding useMSAL to true as workloadidentityfederation supports only MSAL
2025-12-18T19:00:27.2475369Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data subscriptionid = b67c24d2-028d-4610-8370-2d0b793bec40
2025-12-18T19:00:27.2475870Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data subscriptionname = Meru Dev/Test infra
2025-12-18T19:00:27.2477472Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 auth param serviceprincipalid = ***
2025-12-18T19:00:27.2477990Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data activeDirectoryAuthority = https://login.microsoftonline.com/
2025-12-18T19:00:27.2478522Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 auth param tenantid = 72f988bf-86f1-41af-91ab-2d7cd011db47
2025-12-18T19:00:27.2479019Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4=https://management.azure.com/
2025-12-18T19:00:27.2479580Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data environment = AzureCloud
2025-12-18T19:00:27.2481152Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 auth scheme = WorkloadIdentityFederation
2025-12-18T19:00:27.2481649Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data msiclientId = undefined
2025-12-18T19:00:27.2482186Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data activeDirectoryServiceEndpointResourceId = https://management.core.windows.net/
2025-12-18T19:00:27.2482715Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data AzureKeyVaultServiceEndpointResourceId = https://vault.azure.net
2025-12-18T19:00:27.2483218Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data AzureKeyVaultDnsSuffix = vault.azure.net
2025-12-18T19:00:27.2483743Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data ScopeLevel = Subscription
2025-12-18T19:00:27.2484202Z ##[debug]MSAL - getEndpoint - useGraphActiveDirectoryResource=false
2025-12-18T19:00:27.2484640Z ##[debug]MSAL - getEndpoint - useMSAL=true
2025-12-18T19:00:27.2485425Z ##[debug]MSAL - getEndpoint - endpoint={"subscriptionID":"b67c24d2-028d-4610-8370-2d0b793bec40","subscriptionName":"Meru Dev/Test infra","servicePrincipalClientID":"","environmentAuthorityUrl":"https://login.microsoftonline.com/","tenantID":"72f988bf-86f1-41af-91ab-2d7cd011db47","url":"https://management.azure.com/","environment":"AzureCloud","scheme":"WorkloadIdentityFederation","activeDirectoryResourceID":"https://management.core.windows.net/","azureKeyVaultServiceEndpointResourceId":"https://vault.azure.net","azureKeyVaultDnsSuffix":"vault.azure.net","scopeLevel":"Subscription"}
2025-12-18T19:00:27.2486194Z ##[debug]MSAL - getEndpoint - connectedServiceName=07fbf840-41d8-4a46-a17e-2eac8983aaa4
2025-12-18T19:00:27.2486671Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 auth param authenticationType = undefined
2025-12-18T19:00:27.2487154Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 data EnableAdfsAuthentication = false
2025-12-18T19:00:27.2487788Z ##[debug]Feature 'ALLOWSCOPELEVELTOKEN' not found. Returning false as default.
2025-12-18T19:00:27.2488259Z ##[debug]07fbf840-41d8-4a46-a17e-2eac8983aaa4 auth param apitoken = undefined
2025-12-18T19:00:27.2489448Z ##[debug]{"subscriptionID":"b67c24d2-028d-4610-8370-2d0b793bec40","subscriptionName":"Meru Dev/Test infra","servicePrincipalClientID":"","environmentAuthorityUrl":"https://login.microsoftonline.com/","tenantID":"72f988bf-86f1-41af-91ab-2d7cd011db47","url":"https://management.azure.com/","environment":"AzureCloud","scheme":"WorkloadIdentityFederation","activeDirectoryResourceID":"https://management.azure.com/","azureKeyVaultServiceEndpointResourceId":"https://vault.azure.net","azureKeyVaultDnsSuffix":"vault.azure.net","scopeLevel":"Subscription","isADFSEnabled":false,"applicationTokenCredentials":{"connectedServiceName":"07fbf840-41d8-4a46-a17e-2eac8983aaa4","clientId":"***","tenantId":"72f988bf-86f1-41af-91ab-2d7cd011db47","baseUrl":"https://management.azure.com/","authorityUrl":"https://login.microsoftonline.com/","activeDirectoryResourceId":"https://management.azure.com/","isAzureStackEnvironment":false,"scheme":2,"isADFSEnabled":false,"useMSAL":true,"allowScopeLevelToken":false,"tokenMutex":{"_semaphore":{"_value":1,"_cancelError":{},"_weightedQueues":[],"_weightedWaiters":[]}}}}
2025-12-18T19:00:27.2517502Z SubscriptionId: b67c24d2-028d-4610-8370-2d0b793bec40.
2025-12-18T19:00:27.2518124Z Key vault name: merucdtestinraakv.
2025-12-18T19:00:27.2519234Z ##[debug]set SYSTEM_UNSAFEALLOWMULTILINESECRET=true
2025-12-18T19:00:27.2531980Z ##[debug]Processed: ##vso[task.setvariable variable=SYSTEM_UNSAFEALLOWMULTILINESECRET;isOutput=false;issecret=false;]true
2025-12-18T19:00:27.2533468Z ##[debug]Downloading all secrets from subscriptionId: b67c24d2-028d-4610-8370-2d0b793bec40, vault: merucdtestinraakv
2025-12-18T19:00:27.2534185Z Downloading secrets using: https://merucdtestinraakv.vault.azure.net/secrets?maxresults=25&api-version=2016-10-01.
2025-12-18T19:00:27.2535022Z ##[debug]MSAL - getMSALToken called. force=undefined
2025-12-18T19:00:27.2535816Z ##[debug]system.debug=true
2025-12-18T19:00:27.2536587Z ##[debug]system.debug=true
2025-12-18T19:00:27.2537344Z ##[debug]agent.proxyurl=undefined
2025-12-18T19:00:27.2538144Z ##[debug]agent.proxybypasslist=undefined
2025-12-18T19:00:27.2538936Z ##[debug]MSAL - FederatedAccess - OIDC is used.
2025-12-18T19:00:27.2539759Z ##[debug]System.TeamProjectId=5603b9e3-ece0-4518-b9c8-1a36054d9970
2025-12-18T19:00:27.2540601Z ##[debug]System.HostType=release
2025-12-18T19:00:27.2541420Z ##[debug]System.PlanId=1164baa2-a83f-4a7c-bfe6-1bdd3e9f9a25
2025-12-18T19:00:27.2542273Z ##[debug]System.JobId=72cc25b3-bb52-5963-8471-48dad4de472e
2025-12-18T19:00:27.2543137Z ##[debug]System.CollectionUri=https://msdata.vsrm.visualstudio.com/
2025-12-18T19:00:27.2544067Z ##[debug]Getting credentials for local feeds
2025-12-18T19:00:27.2544883Z ##[debug]SYSTEMVSSCONNECTION exists true
2025-12-18T19:00:27.2545651Z ##[debug]Got auth token
2025-12-18T19:00:27.2550615Z ##[debug]Feature 'UseOIDCToken2InAzureArmRest' = 'True'. Processed as 'true'.
2025-12-18T19:00:28.3030641Z ##[error]Error while trying to get OIDC token: Error: 'Stage_5603b9e3ece04518b9c81a36054d9970_15201_4.Phase_1.Multiplier: OrcasBreadth.DirectorV2,wasd-test-ossmc-southeastasia1-d' is not a valid node instance.
2025-12-18T19:00:28.3036861Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=63162649-1804-4cdf-8298-b02dfc0c6632;]Error while trying to get OIDC token: Error: 'Stage_5603b9e3ece04518b9c81a36054d9970_15201_4.Phase_1.Multiplier: OrcasBreadth.DirectorV2,wasd-test-ossmc-southeastasia1-d' is not a valid node instance.
2025-12-18T19:00:28.3037927Z ##[debug]Retrying OIDC token fetch. Retries left: 2
2025-12-18T19:00:30.8548057Z ##[error]Error while trying to get OIDC token: Error: 'Stage_5603b9e3ece04518b9c81a36054d9970_15201_4.Phase_1.Multiplier: OrcasBreadth.DirectorV2,wasd-test-ossmc-southeastasia1-d' is not a valid node instance.
2025-12-18T19:00:30.8549618Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=63162649-1804-4cdf-8298-b02dfc0c6632;]Error while trying to get OIDC token: Error: 'Stage_5603b9e3ece04518b9c81a36054d9970_15201_4.Phase_1.Multiplier: OrcasBreadth.DirectorV2,wasd-test-ossmc-southeastasia1-d' is not a valid node instance.
2025-12-18T19:00:30.8550389Z ##[debug]Retrying OIDC token fetch. Retries left: 1
2025-12-18T19:00:35.3782872Z ##[error]Error while trying to get OIDC token: Error: 'Stage_5603b9e3ece04518b9c81a36054d9970_15201_4.Phase_1.Multiplier: OrcasBreadth.DirectorV2,wasd-test-ossmc-southeastasia1-d' is not a valid node instance.
2025-12-18T19:00:35.3784047Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=63162649-1804-4cdf-8298-b02dfc0c6632;]Error while trying to get OIDC token: Error: 'Stage_5603b9e3ece04518b9c81a36054d9970_15201_4.Phase_1.Multiplier: OrcasBreadth.DirectorV2,wasd-test-ossmc-southeastasia1-d' is not a valid node instance.
2025-12-18T19:00:35.3784738Z ##[debug]Retrying OIDC token fetch. Retries left: 0
2025-12-18T19:00:42.3211424Z ##[error]Error while trying to get OIDC token: Error: 'Stage_5603b9e3ece04518b9c81a36054d9970_15201_4.Phase_1.Multiplier: OrcasBreadth.DirectorV2,wasd-test-ossmc-southeastasia1-d' is not a valid node instance.
2025-12-18T19:00:42.3212555Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=63162649-1804-4cdf-8298-b02dfc0c6632;]Error while trying to get OIDC token: Error: 'Stage_5603b9e3ece04518b9c81a36054d9970_15201_4.Phase_1.Multiplier: OrcasBreadth.DirectorV2,wasd-test-ossmc-southeastasia1-d' is not a valid node instance.
2025-12-18T19:00:42.3213899Z ##[debug]{}
2025-12-18T19:00:42.3216562Z ##[debug]task result: Failed
2025-12-18T19:00:42.3217331Z ##[error]Get secrets failed. Error: Could not fetch access token for AAD. Retry limit exceeded..
2025-12-18T19:00:42.3217879Z ##[debug]Processed: ##vso[task.issue type=error;source=TaskInternal;correlationId=63162649-1804-4cdf-8298-b02dfc0c6632;]Get secrets failed. Error: Could not fetch access token for AAD. Retry limit exceeded..
2025-12-18T19:00:42.3219962Z ##[debug]Processed: ##vso[task.complete result=Failed;]Get secrets failed. Error: Could not fetch access token for AAD. Retry limit exceeded..
2025-12-18T19:00:42.3342120Z ##[section]Finishing: Azure Key Vault: merucdtestinraakv