Axway Agent Plugin Installation Fails to Enable OTel Export to Local Dynatrace OneAgent

[Leonardobmuller]

Installation of the Axway Agent to send data to the local OneAgent didn't work. OTel was not enabled after the plugin installation. Axway Support was also unable to install it in their environment and received the following error:

java.lang.NoClassDefFoundError: io/opentelemetry/instrumentation/api/internal/EmbeddedInstrumentationProperties

[juliusloman]

Hi @Leonardobmuller ,

You are likely missing the opentelemetry-instrumentation-api library. Recently, I've worked on a fork of this repository to make it more suitable for Dynatrace deployments. Other changes were alignment with OTEL semantic dictionary and automatic detection of APIs / http routes, so that you can fully monitor individual APIs, endpoints. And of course, to make the OTEL collector optional if you have Dynatrace OneAgent locally deployed and add the necessary resource attributes so that service is connected to process groups/hosts for both SmartScape and licensing purposes.

[Leonardobmuller]

Thanks for having a look at it Julius! Can you spot anything wrong on it? this is what we added on our jvm.xml

These are in-placed libraries related to OTel: INSTALLATION_DIR/ext/lib

-rw-r-----. 1 axwayd axwayd 2.1M Apr 23 12:35 aspectjweaver-1.9.22.1.jar
-rw-r-----. 1 axwayd axwayd 1.7M Apr 16 15:56 kotlin-stdlib-1.9.25.jar
-rw-r-----. 1 axwayd axwayd 772K Apr 16 15:56 okhttp-4.12.0.jar
-rw-r-----. 1 axwayd axwayd 352K Apr 16 15:56 okio-jvm-3.6.0.jar
-rw-r-----. 1 axwayd axwayd 154K Apr 16 15:56 opentelemetry-api-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  63K Apr 16 15:56 opentelemetry-api-incubator-1.42.1-alpha.jar
-rw-r-----. 1 axwayd axwayd  17K May 21 16:50 opentelemetry-apim-agent-0.0.3.jar
-rw-r-----. 1 axwayd axwayd  47K Apr 16 15:56 opentelemetry-context-1.42.1.jar
-rw-r-----. 1 axwayd axwayd 105K Apr 16 15:56 opentelemetry-exporter-common-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  16K Apr 16 15:56 opentelemetry-exporter-logging-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  77K Apr 16 15:56 opentelemetry-exporter-otlp-1.42.1.jar
-rw-r-----. 1 axwayd axwayd 198K Apr 16 15:56 opentelemetry-exporter-otlp-common-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  27K Apr 16 15:56 opentelemetry-exporter-sender-okhttp-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  15K Apr 16 15:56 opentelemetry-extension-incubator-1.30.1-alpha.jar
-rw-r-----. 1 axwayd axwayd 226K May 19 16:36 opentelemetry-instrumentation-api-2.18.1.jar
-rw-r-----. 1 axwayd axwayd  38K Apr 16 15:56 opentelemetry-runtime-telemetry-java8-2.18.1-alpha.jar
-rw-r-----. 1 axwayd axwayd 6.7K Apr 16 15:56 opentelemetry-sdk-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  54K Apr 16 15:56 opentelemetry-sdk-common-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  47K Apr 16 15:56 opentelemetry-sdk-extension-autoconfigure-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  19K Apr 16 15:56 opentelemetry-sdk-extension-autoconfigure-spi-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  53K Apr 16 15:56 opentelemetry-sdk-logs-1.42.1.jar
-rw-r-----. 1 axwayd axwayd 303K Apr 16 15:56 opentelemetry-sdk-metrics-1.42.1.jar
-rw-r-----. 1 axwayd axwayd 128K Apr 16 15:56 opentelemetry-sdk-trace-1.42.1.jar
-rw-r-----. 1 axwayd axwayd  74K Apr 16 15:56 opentelemetry-semconv-1.27.0-alpha.jar

Env. variables: export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:14499/otlp/v1/traces
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_TRACES_EXPORTER=otlp
export OTEL_SERVICE_NAME=apim-gw
export OTEL_METRICS_EXPORTER=none

I was also wondering if I need to add resource attributes, if so I haven't tested with it yet.

OTel from Axway to local OneAgent already installed on the VM's

Thank you

[juliusloman]

Regarding the missing class, I have no idea TBH. The class is in opentelemetry-instrumentation-api-2.18.1.jar which you have. The JVM.xml should contain just the link to aspectjwever similar to:

    <VMArg name="-javaagent:/home/axway/Axway-7.7.0/apigateway/ext/lib/aspectjweaver-1.9.22.1.jar"/>

Over for OneAgent ingestion, I'm using this config, but this should be equivalent to yours.
OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:14499/otlp"
OTEL_EXPORTER_OTLP_PROTOCOL="http/protobuf"
OTEL_TRACES_EXPORTER="otlp"
OTEL_METRICS_EXPORTER="none"
OTEL_SERVICE_NAME="APIGW"

I highly recommend adding resource attributes. In the case you are sending it locally through OneAgent, the license aspect is covered (traces will be included in your OneAgent license), otherwise the trace volume is subject to billing in Dynatrace.
Anyway, I recommend adding the process group instance ID to have a proper link to the process group instance.

This is included in my fork and the metadata is loaded automatically - https://github.com/juliusloman/axway-opentelemetry-apim-agent/blob/feature/otel/src/main/java/com/axway/apim/opentelemetry/Configuration.java#L33