Bug: Deployment > Environment Validator requires ICMP reachability to default gateway for Management IP readiness, but this is undocumented

[schmittnieto]

Environment Validator requires ICMP reachability to default gateway for Management IP readiness, but this is undocumented

Bug description

During a new Azure Local deployment the Environment Validator fails AzureLocal_Network_Test_Infra_IP_Connection_IPReadiness on a single node virtual system when ICMP echo to the default gateway is blocked. The addresses under test are the Management IPs defined earlier in deployment for the Cluster and the Arc Resource Bridge.
The requirement to ping the default gateway is not stated in the documentation (Firewall Requirements for Azure Local) and in previous versions deployment validation succeeded without needing ICMP to the gateway.

Repro steps

1. Prepare a new deployment and define Management IPs for Cluster and Arc Resource Bridge
2. Ensure the default gateway does not respond to ICMP echo from the host during validation
3. Run Environment Validator → Network step
4. Observe the validation failure on the Management IP readiness check

Expected behavior

Validation should not require ICMP to the default gateway in order to mark Management IP readiness as successful, unless the deployment documentation explicitly lists this as a prerequisite. At minimum the behavior and remediation should be updated to clearly state the dependency on ICMP echo to the default gateway.

Environment (please complete the following information)

Build: 12.2510.1002.94
System type: Single node virtual system
Environment: Non production
Region: West Europe
Deployment date: 10 Nov 2025

Screenshots

Not applicable. A shortened log excerpt is included below without personal data.

Correlation ID

If needed I can perform standalone log collection and provide the correlation ID.

Additional context

The failing check is implemented in:

• AzStackHci.EnvironmentChecker\AzStackHciNetwork\AzStackHci.Network.psm1 which calls
• AzStackHci.EnvironmentChecker\AzStackHciNetwork\AzStackHci.Network.Helpers.psm1

In Helpers, Test-NwkValidator_InfraIpPoolReadiness treats the Management IP as ready only after a successful ICMP echo to the default gateway. Relevant code block follows.

# Helpers: Test-NwkValidator_InfraIpPoolReadiness (lines 1600–1660)
$ipStopWatch = [System.diagnostics.stopwatch]::StartNew()
while (-not $currentIPReady -and ($ipStopWatch.Elapsed.TotalSeconds -lt 60))
{
    $ipConfig = Get-NetIPAddress -InterfaceAlias $newVNICName -ErrorAction SilentlyContinue | Where-Object { $_.IPAddress -eq $ipToCheck -and $_.PrefixOrigin -eq "Manual" -and $_.AddressFamily -eq "IPv4" -and $_.AddressState -eq "Preferred" }

    if ($ipConfig)
    {
        # After IP configured on the adapter, will need to try ping from the IP to default gateway to make sure the IP is really ready to use
        Log-Info "Validating ICMP connection from $ipToCheck to default gateway $defaultGateway..."

        $tmpPingSuccess = InvokePingWithRetries -Destination $defaultGateway -Source $ipToCheck -RetryCount 15 -SleepSeconds 1
        if ($tmpPingSuccess) {
            Log-Info "ICMP connection from $ipToCheck to default gateway $defaultGateway is successful."
            $currentIPReady = $true
            break
        }
    }

    Start-Sleep -Seconds 3
}
#endregion
} catch {
    Log-Info "Got exception when trying to set IP $ipToCheck on vNIC $newVNICName."
    $currentIPReady = $false
}
#endregion

if (-not $currentIPReady) {
    Log-Info "Cannot get the IP $ipToCheck ready on the vNIC $newVNICName. Skip to next IP."

    $infraIpNotReadyRstParams = @{
        Name               = 'AzureLocal_Network_Test_Infra_IP_Connection_IPReadiness'
        Title              = 'Test IP readiness on test adapter for IP from infra pool'
        DisplayName        = 'Test IP readiness on test adapter for IP from infra pool'
        Severity           = 'CRITICAL'
        Description        = 'Test IP readiness on test adapter for IP from infra pool'
        Tags               = @{}
        Remediation        = "Make sure infra IP $ipToCheck is routable to your gateway $defaultGateway, and the IP is not used by any other device or service on the network."
        TargetResourceID   = "Infra_IP_Connection_$($ipToCheck)"
        TargetResourceName = "Infra_IP_Connection_$($ipToCheck)"
        TargetResourceType = "Infra_IP_Connection_$($ipToCheck)"
        Timestamp          = [datetime]::UtcNow
        Status             = "FAILURE"
        AdditionalData     = @{
            Source    = $env:COMPUTERNAME
            Resource  = $($ipToCheck)
            Detail    = "[FAILED] Connection from $ipToCheck to gateway $defaultGateway failed. Cannot get the IP configured correctly on the test adapter."
            Status    = "FAILURE"
            TimeStamp = [datetime]::UtcNow
        }
        HealthCheckSource  = $ENV:EnvChkrId
    }

    $instanceResults += New-AzStackHciResultObject @infraIpNotReadyRstParams
    continue
} else {
    Log-Info "IP $ipToCheck ready on the vNIC $newVNICName."
}

Shortened log excerpt showing a single failing entry:

2025-11-10 08:29:43 Verbose  [EnvironmentValidator:ValidateNetwork] Validator failed. {
  "ExceptionType": "json",
  "ErrorMessage": {
    "Message": "Network requirements not met. Review output and remediate.",
    "Results": [
      {
        "Name": "AzureLocal_Network_Test_Infra_IP_Connection_IPReadiness",
        "DisplayName": "Test IP readiness on test adapter for IP from infra pool",
        "Title": "Test IP readiness on test adapter for IP from infra pool",
        "Status": 1,
        "Severity": 2,
        "Description": "Test IP readiness on test adapter for IP from infra pool",
        "Remediation": "Make sure infra IP 172.19.19.20 is routable to your gateway 172.19.19.1, and the IP is not used by any other device or service on the network.",
        "TargetResourceID": "Infra_IP_Connection_172.19.19.20",
        "TargetResourceName": "Infra_IP_Connection_172.19.19.20",
        "TargetResourceType": "Infra_IP_Connection_172.19.19.20",
        "Timestamp": "/Date(1762762838119)/",
        "AdditionalData": {
          "Detail": "[FAILED] Connection from 172.19.19.20 to gateway 172.19.19.1 failed. Cannot get the IP configured correctly on the test adapter.",
          "Status": "FAILURE",
          "TimeStamp": "11/10/2025 08:20:38",
          "Resource": "172.19.19.20",
          "Source": "AZLNODE01"
        },
        "HealthCheckSource": "Deployment\\Standard\\Small\\Network\\4572f390"
      }
    ]
  },
  "ExceptionStackTrace": "at Test-AzStackHciNetwork, C:\\Program Files\\WindowsPowerShell\\Modules\\AzStackHci.EnvironmentChecker\\AzStackHciNetwork\\AzStackHciNetwork.psm1: line 236"
}

[schmittnieto]

Hi @AlBurns-MSFT, quick addendum for clarity:

• The environment had full internet connectivity, DNS resolution, and outbound access to required endpoints. The only thing blocked by policy was ICMP echo to the default gateway.
• The failure occurs during Test-NwkValidator_InfraIpPoolReadiness, which treats a Management IP as ready only after a successful ping to the default gateway. You can reproduce by denying ICMP echo on the gateway while keeping routing and DNS intact.
• Versions in use: build 12.2510.1002.94 and AzStackHci.EnvironmentChecker.Deploy 10.2510.0.2100.

If you need reproducibility I am happy to provide a sanitized log bundle and correlation ID or join a short live repro. I do not think a PR is the best path to gain visibility here since this is about validator behavior versus documented prerequisites. Feel free to reach out to me directly.

Thanks, and have a great start to the week.

PS: Quick note on why I tagged you here. You authored the earlier PR #172 that fixed a similar validator behavior, so I wanted to flag this to you first. Please feel free to route this to the right owner or ignore it if it is not in your scope.