Skip to content

fix(mistral): report usage metrics in streaming mode#1697

Open
jackatorcflo wants to merge 1 commit intostrands-agents:mainfrom
jackatorcflo:fix/mistral-streaming-token-usage
Open

fix(mistral): report usage metrics in streaming mode#1697
jackatorcflo wants to merge 1 commit intostrands-agents:mainfrom
jackatorcflo:fix/mistral-streaming-token-usage

Conversation

@jackatorcflo
Copy link

@jackatorcflo jackatorcflo commented Feb 15, 2026

Description

Fixed a bug where Mistral models were not reporting usage metrics (prompt tokens, completion tokens, total tokens) when used in streaming mode.

The root cause was that the streaming response handler checked for usage data at chunk.usage (top-level), but Mistral's v1+ SDK returns all streaming data nested under a data property. According to the Mistral Python SDK migration guide, streaming chunks are accessed via chunk.data.choices[0].delta.content with the additional .data wrapper layer. This same nesting applies to the usage field, making the correct path chunk.data.usage.

This is confirmed by the Langfuse Mistral integration example, which shows the correct access pattern:

if chunk.data.choices[0].finish_reason == "stop":
    usage = {
        "input": chunk.data.usage.prompt_tokens,
        "output": chunk.data.usage.completion_tokens
    }

This bug manifested silently; Mistral model streaming worked, but we noticed that we never received token data for streaming Mistral models.

Related Issues

  • N/A

Documentation PR

  • N/A

Type of Change

  • Bug fix

Testing

How have you tested the change? Verify that the changes do not break functionality or introduce warnings in consuming repositories: agents-docs, agents-tools, agents-cli

  • I manually tested this before and after:

    • Before: Usage data for streaming Mistral models was empty (0 input tokens, 0 completion tokens).
    • After: Usage data for streaming Mistral models was correct (correct input tokens, correct completed tokens).
  • I updated the Mistral tests to mock the usage at the correct level (under data).

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@github-actions
Copy link

Assessment: Approve

This is a clean, well-documented bug fix that correctly addresses the Mistral SDK v1+ API structure change for accessing usage data in streaming mode.

Review Summary
  • Correctness: The fix properly accesses chunk.data.usage instead of chunk.usage, matching the documented Mistral SDK v1+ structure where streaming data is nested under a data property.
  • Defensive coding: The condition hasattr(chunk, "data") and hasattr(chunk.data, "usage") and chunk.data.usage appropriately handles edge cases where the data may not be present.
  • Tests: Mock structures are correctly updated to reflect the actual API, following existing test patterns in the codebase.

Good job on the thorough PR description with external documentation references! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant