Skip to content

Conversation

@ameyjain
Copy link
Contributor

@ameyjain ameyjain commented Jan 15, 2026

Export evaluator LLM call spans with the same trace_id as agent execution spans. This allows both agent and evaluator spans to be fetched together using a single ID.

Changes:

  • Add evaluator_spans field to EvalRunUpdatedEvent
  • Collect evaluator spans after all evaluators run in _execute_eval
  • Export evaluator spans with the correct trace_id in handle_update_eval_run

Development Package

  • Use uipath pack --nolock to get the latest dev build from this PR (requires version range).
  • Add this package as a dependency in your pyproject.toml:
[project]
dependencies = [
  # Exact version:
  "uipath==2.5.4.dev1011303908",

  # Any version from PR
  "uipath>=2.5.4.dev1011300000,<2.5.4.dev1011310000"
]

[[tool.uv.index]]
name = "testpypi"
url = "https://test.pypi.org/simple/"
publish-url = "https://test.pypi.org/legacy/"
explicit = true

[tool.uv.sources]
uipath = { index = "testpypi" }

[tool.uv]
override-dependencies = [
    "uipath>=2.5.4.dev1011300000,<2.5.4.dev1011310000",
]

BEFORE

Screenshot 2026-01-15 at 4 26 26 PM

After

Screenshot 2026-01-15 at 4 25 21 PM

@ameyjain ameyjain added the build:dev Create a dev build from the pr label Jan 15, 2026
@github-actions github-actions bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository labels Jan 15, 2026
@ameyjain ameyjain removed build:dev Create a dev build from the pr test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository labels Jan 15, 2026
@github-actions github-actions bot added test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository labels Jan 15, 2026
@ameyjain ameyjain changed the title feat: export evaluator spans with eval_run_id trace_id feat: export evaluator spans Jan 15, 2026
@ameyjain ameyjain force-pushed the feat/evaluator-spans-tracing branch from 1217434 to cf5ad68 Compare January 15, 2026 22:05
@AAgnihotry
Copy link
Contributor

@ameyjain how did you get these 2 traces ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:uipath-langchain Triggers tests in the uipath-langchain-python repository test:uipath-llamaindex Triggers tests in the uipath-llamaindex-python repository

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants