Skip to content

GenAI Utils | Agent metric demo#4343

Closed
etserend wants to merge 3 commits intoopen-telemetry:mainfrom
etserend:genai-utils/agent-metric-demo
Closed

GenAI Utils | Agent metric demo#4343
etserend wants to merge 3 commits intoopen-telemetry:mainfrom
etserend:genai-utils/agent-metric-demo

Conversation

@etserend
Copy link
Copy Markdown

Description

Adds a combined demo application that exercises the full agent lifecycle with OpenTelemetry telemetry: agent creation, deployment to Vertex AI Agent Engine, agent invocation with auto-instrumented LLM child spans, and metric/event emission.

Sample metrics output:

Metrics — gen_ai.client.operation.duration (Histogram, unit: s)
├── Data Point 1 — create_agent (local, ~0ms)
│   ├── gen_ai.operation.name: create_agent
│   ├── gen_ai.request.model: gemini-2.5-flash
│   ├── gen_ai.provider.name: gcp_vertex_ai
│   └── server.address: us-central1-aiplatform.googleapis.com
│
├── Data Point 2 — create_agent (deploy, ~259.75s)
│   ├── gen_ai.operation.name: create_agent
│   ├── gen_ai.request.model: gemini-2.5-flash
│   ├── gen_ai.provider.name: gcp_vertex_ai
│   └── server.address: us-central1-aiplatform.googleapis.com
│
└── Data Point 3 — invoke_agent (~3.16s)
    ├── gen_ai.operation.name: invoke_agent
    ├── gen_ai.request.model: gemini-2.5-flash
    ├── gen_ai.provider.name: gcp_vertex_ai
    └── server.address: us-central1-aiplatform.googleapis.com

Metrics — gen_ai.client.token.usage (Histogram, unit: {token})
├── Data Point 1 — input tokens (invoke_agent)
│   ├── gen_ai.operation.name: invoke_agent
│   ├── gen_ai.request.model: gemini-2.5-flash
│   ├── gen_ai.provider.name: gcp_vertex_ai
│   ├── gen_ai.token.type: input
│   ├── server.address: us-central1-aiplatform.googleapis.com
│   └── value: 12
│
└── Data Point 2 — output tokens (invoke_agent)
    ├── gen_ai.operation.name: invoke_agent
    ├── gen_ai.request.model: gemini-2.5-flash
    ├── gen_ai.provider.name: gcp_vertex_ai
    ├── gen_ai.token.type: output
    ├── server.address: us-central1-aiplatform.googleapis.com
    └── value: 105
        

Sample events output:

Events — gen_ai.client.inference.operation.details (3 log records)
│
├── Event 1 — create_agent (trace: e7f8a7cd..., scope: opentelemetry.util.genai.handler v0.3b0.dev)
│   ├── gen_ai.operation.name: create_agent
│   ├── gen_ai.agent.name: Currency Exchange Agent
│   ├── gen_ai.agent.description: Deploying agent to Vertex AI Agent Engine
│   ├── gen_ai.provider.name: gcp_vertex_ai
│   ├── gen_ai.request.model: gemini-2.5-flash
│   └── server.address: us-central1-aiplatform.googleapis.com
│
├── Event 2 — invoke_agent (trace: 4dc8827b..., scope: opentelemetry.util.genai.handler v0.3b0.dev)
│   ├── gen_ai.operation.name: invoke_agent
│   ├── gen_ai.agent.name: Currency Exchange Agent
│   ├── gen_ai.agent.description: Currency exchange agent demo
│   ├── gen_ai.provider.name: gcp_vertex_ai
│   ├── gen_ai.request.model: gemini-2.5-flash
│   ├── gen_ai.response.finish_reasons:
│   ├── gen_ai.usage.input_tokens: 12
│   ├── gen_ai.usage.output_tokens: 105
│   └── server.address: us-central1-aiplatform.googleapis.com
│
└── Event 3 — chat (trace: 4dc8827b..., scope: opentelemetry.instrumentation.vertexai)
    ├── gen_ai.operation.name: chat
    ├── gen_ai.request.model: gemini-2.5-flash
    ├── gen_ai.response.model: gemini-2.5-flash
    ├── gen_ai.response.finish_reasons:
    ├── gen_ai.usage.input_tokens: 12
    ├── gen_ai.usage.output_tokens: 105
    ├── gen_ai.input.messages: [{"parts":[{"content":"What is the ex...
    ├── gen_ai.output.messages: [{"finish_reason":"stop","parts":[{"...
    ├── server.address: us-central1-aiplatform.googleapis.com
    └── server.port: 443

Fixes # (issue)

Type of change

  • New feature (non-breaking change which adds functionality)

Does This PR Require a Core Repo Change?

  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 2, 2026

This PR has been automatically marked as stale because it has not had any activity for 14 days. It will be closed if no further activity occurs within 14 days of this comment.
If you're still working on this, please add a comment or push new commits.

@github-actions github-actions Bot added the Stale label Apr 2, 2026
@github-actions
Copy link
Copy Markdown

This PR has been closed due to inactivity. Please reopen if you would like to continue working on it.

@github-actions github-actions Bot closed this Apr 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants