fix(core): Improve Vercel AI SDK instrumentation attributes#19717
Open
RulaKhaled wants to merge 11 commits intodevelopfrom
Open
fix(core): Improve Vercel AI SDK instrumentation attributes#19717RulaKhaled wants to merge 11 commits intodevelopfrom
RulaKhaled wants to merge 11 commits intodevelopfrom
Conversation
Contributor
size-limit report 📦
|
Contributor
node-overhead report 🧳Note: This is a synthetic benchmark with a minimal express app and does not necessarily reflect the real-world performance impact in an application.
|
RulaKhaled
changed the title
RulaKhaled
changed the title
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: V6 tests missing new output messages attribute assertions
- Added explicit
GEN_AI_OUTPUT_MESSAGES_ATTRIBUTEassertions (and import) across the v6 span expectations sogen_ai.output.messagesis now validated for text and tool-call outputs.
- Added explicit
Or push these changes by commenting:
@cursor push 8e0d6cceb7
Preview (8e0d6cceb7)
diff --git a/dev-packages/node-integration-tests/suites/tracing/vercelai/v6/test.ts b/dev-packages/node-integration-tests/suites/tracing/vercelai/v6/test.ts
--- a/dev-packages/node-integration-tests/suites/tracing/vercelai/v6/test.ts
+++ b/dev-packages/node-integration-tests/suites/tracing/vercelai/v6/test.ts
@@ -4,6 +4,7 @@
import {
GEN_AI_INPUT_MESSAGES_ATTRIBUTE,
GEN_AI_OPERATION_NAME_ATTRIBUTE,
+ GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE,
GEN_AI_REQUEST_AVAILABLE_TOOLS_ATTRIBUTE,
GEN_AI_REQUEST_MODEL_ATTRIBUTE,
GEN_AI_RESPONSE_FINISH_REASONS_ATTRIBUTE,
@@ -97,6 +98,8 @@
'vercel.ai.settings.maxRetries': 2,
'vercel.ai.streaming': false,
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: '[{"role":"user","content":"Where is the second span?"}]',
+ [GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
+ '[{"role":"assistant","parts":[{"type":"text","content":"Second span here!"}],"finish_reason":"stop"}]',
[GEN_AI_RESPONSE_MODEL_ATTRIBUTE]: 'mock-model-id',
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 10,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 20,
@@ -129,6 +132,8 @@
'vercel.ai.response.id': expect.any(String),
'vercel.ai.response.timestamp': expect.any(String),
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: expect.any(String),
+ [GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
+ '[{"role":"assistant","parts":[{"type":"text","content":"Second span here!"}],"finish_reason":"stop"}]',
[GEN_AI_RESPONSE_FINISH_REASONS_ATTRIBUTE]: ['stop'],
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 10,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 20,
@@ -231,6 +236,8 @@
'vercel.ai.prompt': '[{"role":"user","content":"Where is the first span?"}]',
'vercel.ai.request.headers.user-agent': expect.any(String),
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: '[{"role":"user","content":"Where is the first span?"}]',
+ [GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
+ '[{"role":"assistant","parts":[{"type":"text","content":"First span here!"}],"finish_reason":"stop"}]',
'vercel.ai.response.finishReason': 'stop',
'vercel.ai.settings.maxRetries': 2,
'vercel.ai.streaming': false,
@@ -257,6 +264,8 @@
'vercel.ai.request.headers.user-agent': expect.any(String),
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]:
'[{"role":"user","content":[{"type":"text","text":"Where is the first span?"}]}]',
+ [GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
+ '[{"role":"assistant","parts":[{"type":"text","content":"First span here!"}],"finish_reason":"stop"}]',
'vercel.ai.response.finishReason': 'stop',
'vercel.ai.response.id': expect.any(String),
'vercel.ai.response.model': 'mock-model-id',
@@ -289,6 +298,8 @@
'vercel.ai.prompt': '[{"role":"user","content":"Where is the second span?"}]',
'vercel.ai.request.headers.user-agent': expect.any(String),
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: '[{"role":"user","content":"Where is the second span?"}]',
+ [GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
+ '[{"role":"assistant","parts":[{"type":"text","content":"Second span here!"}],"finish_reason":"stop"}]',
'vercel.ai.response.finishReason': 'stop',
'vercel.ai.settings.maxRetries': 2,
'vercel.ai.streaming': false,
@@ -324,6 +335,8 @@
'vercel.ai.response.id': expect.any(String),
'vercel.ai.response.timestamp': expect.any(String),
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: expect.any(String),
+ [GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
+ '[{"role":"assistant","parts":[{"type":"text","content":"Second span here!"}],"finish_reason":"stop"}]',
[GEN_AI_RESPONSE_FINISH_REASONS_ATTRIBUTE]: ['stop'],
[GEN_AI_USAGE_INPUT_TOKENS_ATTRIBUTE]: 10,
[GEN_AI_USAGE_OUTPUT_TOKENS_ATTRIBUTE]: 20,
@@ -346,6 +359,8 @@
'vercel.ai.prompt': '[{"role":"user","content":"What is the weather in San Francisco?"}]',
'vercel.ai.request.headers.user-agent': expect.any(String),
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: '[{"role":"user","content":"What is the weather in San Francisco?"}]',
+ [GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
+ '[{"role":"assistant","parts":[{"type":"tool_call","id":"call-1","name":"getWeather","arguments":"{\\"location\\":\\"San Francisco\\"}"}],"finish_reason":"tool-calls"}]',
'vercel.ai.response.finishReason': 'tool-calls',
'vercel.ai.settings.maxRetries': 2,
'vercel.ai.streaming': false,
@@ -371,6 +386,8 @@
'vercel.ai.pipeline.name': 'generateText.doGenerate',
'vercel.ai.request.headers.user-agent': expect.any(String),
[GEN_AI_INPUT_MESSAGES_ATTRIBUTE]: expect.any(String),
+ [GEN_AI_OUTPUT_MESSAGES_ATTRIBUTE]:
+ '[{"role":"assistant","parts":[{"type":"tool_call","id":"call-1","name":"getWeather","arguments":"{\\"location\\":\\"San Francisco\\"}"}],"finish_reason":"tool-calls"}]',
'vercel.ai.prompt.toolChoice': expect.any(String),
[GEN_AI_REQUEST_AVAILABLE_TOOLS_ATTRIBUTE]: EXPECTED_AVAILABLE_TOOLS_JSON,
'vercel.ai.response.finishReason': 'tool-calls',
andreiborza
reviewed
Mar 10, 2026
Comment on lines
+134
to
+140
| const toolName = span.data[GEN_AI_TOOL_NAME_ATTRIBUTE]; | ||
| if (typeof toolName === 'string') { | ||
| const description = findToolDescription(event.spans, toolName); | ||
| if (description) { | ||
| span.data[GEN_AI_TOOL_DESCRIPTION_ATTRIBUTE] = description; | ||
| } | ||
| } |
Member
There was a problem hiding this comment.
l: Could we extract this into a helper, like we did for applyAccumulatedTokens?
andreiborza
approved these changes
Mar 10, 2026
nicohrubec
reviewed
Mar 10, 2026
| function truncateContentArrayMessage(message: ContentArrayMessage, maxBytes: number): unknown[] { | ||
| const { content } = message; | ||
|
|
||
| // Find the first text part to truncate |
Member
There was a problem hiding this comment.
m: Why do we only truncate the first text part? Is the assumption that these messages usually only have one text part?
Member
Author
There was a problem hiding this comment.
Yes, because this is the most common use case, but we could and should account for more parts, i'll update
| */ | ||
| function normalizeFinishReason(finishReason: unknown): string { | ||
| if (typeof finishReason !== 'string') { | ||
| return 'stop'; |
Member
There was a problem hiding this comment.
l: why do we default to stop if nothing is set?
Member
Author
There was a problem hiding this comment.
because finish_reason is required according to the OTel schema for output messages. https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-output-messages.json
"FinishReason": {
"enum": [
"stop",
"length",
"content_filter",
"tool_call",
"error"
]
}
when the SDK doesn't give us one, 'stop' (normal completion) is the most sensible default assumption.
| // eslint-disable-next-line @typescript-eslint/no-dynamic-delete | ||
| delete attributes[AI_RESPONSE_TEXT_ATTRIBUTE]; | ||
| // eslint-disable-next-line @typescript-eslint/no-dynamic-delete | ||
| delete attributes[AI_RESPONSE_TOOL_CALLS_ATTRIBUTE]; |
Member
There was a problem hiding this comment.
l: we do not delete the original finish reason attribute after normalizing here, is that on purpose?
Member
Author
There was a problem hiding this comment.
yah finish reason is an independent attribute that was not deprecated by output messages attribute https://getsentry.github.io/sentry-conventions/attributes/gen_ai/#gen_ai-response-finish_reasons
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| } else { | ||
| // Subsequent text part doesn't fit: stop here | ||
| break; | ||
| } |
There was a problem hiding this comment.
Non-text parts before text prevent text truncation
Low Severity
In truncateContentArrayMessage, the includedParts.length === 0 check determines whether to truncate a text part that doesn't fit. However, non-text parts are always pushed into includedParts unconditionally. So if a non-text part appears before a text part in the content array (e.g., [image_part, text_part]), includedParts will already be non-empty when the text part is evaluated. This causes the text to be dropped entirely instead of being truncated, unlike the analogous truncatePartsMessage where all parts go through the same size-check path. The result is a content array with only the non-text part and no text at all, even though a truncated text could have fit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces some attributes and fixes to Vercel AI SDK:
Closes #19574