Authenticate to IoT Hub using AAD in tests

[damonbarry]

This change updates the test apps (IotEdgeQuickstart for connectivity, Microsoft.Azure.Devices.Edge.Test for all the others) and test modules used by the test pipelines to:

• replace Microsoft.Azure.EventHubs (deprecated, doesn't support federated credentials) with Azure.Messaging.EventHubs to read messages from Event Hub
• update to the latest Microsoft.Azure.Devices SDK (older version doesn't support federated credentials) to communicate with IoT Hub (working with device registry, twins, edge deployments)

In the test apps, which run on the pipeline agent VM and have access to Azure CLI credentials, we pass Azure.Identity.AzureCliCredential to the SDKs to communicate with Event Hub and IoT Hub.

In the test modules, which run in Docker containers and do not have access to Azure CLI credentials on the pipeline agent VM, but can have credentials mounted as a file, we pass Azure.Identity.WorkloadIdentityCredential to the SDKs to communicate with Event Hub and IoT Hub. In the connectivity test pipeline, which can run for longer than the lifetime of the file-mounted credentials, we use an OIDC API in DevOps to generate and refresh the mounted credential from the system access token passed into the pipeline.

This change also updates the pipeline YAML files and related scripts to support the updates to the test apps.

To test, I ran the various pipelines and confirmed they pass when local auth is disabled on the corresponding IoT hub:

• Connectivity tests
• End-to-end tests (still need to install Azure CLI on arm32v7 physical devices, and update the proxy agent)
• Nested edge end-to-end tests
• ISA-95 smoke tests

Azure IoT Edge PR checklist:

This checklist is used to make sure that common guidelines for a pull request are followed.

General Guidelines and Best Practices

• I have read the contribution guidelines.
• Title of the pull request is clear and informative.
• Description of the pull request includes a concise summary of the enhancement or bug fix.

Testing Guidelines

• Pull request includes test coverage for the included changes.
• Description of the pull request includes
  • concise summary of tests added/modified
  • local testing done.