Skip to content

Conversation

@Hdandria
Copy link

@Hdandria Hdandria commented Dec 4, 2025

Summary

This PR fixes an issue in the langchain-mistralai chat integration where max_retries were not respected when a failure occurred during streaming (e.g., httpx.StreamError after the first chunk).
Yesterday, Mistral was experiencing failure systems (30% of requests were not returning any chunks). I lowered retry to 5 secondes, because 100% of calls not returning any chunks in 5 secondes were bug? And I discovered the retry mechanism was not correctly implemented. This behavior was inconsistent with ChatOpenAI LLM, for exemple.

Changes:

  • Update ChatMistralAI._astream so that the async streaming loop is wrapped in a retry policy based on tenacity.AsyncRetrying.
  • Ensure that httpx.RequestError and httpx.StreamError raised while consuming the SSE stream trigger new attempts, instead of failing after a single try.
  • Add a regression test test_streaming_retry_on_stream_failure using httpx.MockTransport to simulate a mid-stream network failure and assert that multiple HTTP requests are made when retries are enabled.

Rationale

The current Mistral integration uses raw httpx + httpx-sse instead of the official Mistral client, so it must implement its own retry logic. Previously, the retry decorator only wrapped the initial connection. Once the SSE iterator was returned, any StreamError occurring during consumption bypassed the retry logic, resulting in only one failed attempt regardless of the configured max_retries.

This PR preserves the existing public API and behavior while making the streaming path consistent with the max_retries setting.

Testing

Local tests run:

uv run pytest libs/partners/mistralai/tests/unit_tests/test_streaming_retry.py

Future work

The current integration is built directly on top of httpx and httpx-sse. In the long term, it would likely be more robust and easier to maintain to layer ChatMistralAI on top of the official Mistral Python SDK instead of reimplementing low-level HTTP, streaming, and retry behavior by hand. That refactor would be a larger, separate change, but this PR moves the current httpx-based implementation to a more correct and resilient state in the meantime.

AI: Description of PR was made by AI.

@Hdandria Hdandria requested review from ccurme and mdrxy as code owners December 4, 2025 16:57
@github-actions github-actions bot added integration Related to a provider partner package integration mistralai labels Dec 4, 2025
@Hdandria Hdandria changed the title Fixing a bug in the retry of langchain for streaming answer. fix(mistralai): respect max_retries during streaming failures Dec 4, 2025
@github-actions github-actions bot added the fix label Dec 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fix integration Related to a provider partner package integration mistralai

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant