Skip to content

Fix #4478: Detect Azure OpenAI models by endpoint, not just deployment name#4480

Open
devin-ai-integration[bot] wants to merge 1 commit intomainfrom
devin/1770985548-fix-azure-is-openai-model
Open

Fix #4478: Detect Azure OpenAI models by endpoint, not just deployment name#4480
devin-ai-integration[bot] wants to merge 1 commit intomainfrom
devin/1770985548-fix-azure-is-openai-model

Conversation

@devin-ai-integration
Copy link
Contributor

Fix #4478: Detect Azure OpenAI models by endpoint, not just deployment name

Summary

AzureCompletion.is_openai_model was determined solely by checking model name prefixes (gpt-, o1-, text-). Since Azure deployment names can be arbitrary (e.g. gpt5nano), this caused is_openai_model to be False for custom-named deployments, which silently broke:

  • response_model / structured output (JSON schema not sent to API, validation skipped)
  • Tool/function calling (tools not included in request params)
  • supports_function_calling() returning False

Fix: Compute is_azure_openai_endpoint first, then use it as an additional signal — if the endpoint matches openai.azure.com + /openai/deployments/, it's an OpenAI model regardless of deployment name. Non-OpenAI models on Azure AI Inference endpoints (e.g. models.inference.ai.azure.com) are unaffected.

The production code change is 6 lines in AzureCompletion.__init__(). 7 new regression tests added covering detection, response_model parsing, params preparation, tool inclusion, streaming, and negative cases.

Review & Testing Checklist for Human

  • Verify the assumption that openai.azure.com/openai/deployments/ always implies an OpenAI model — could a non-OpenAI model ever be deployed at this endpoint pattern?
  • Ideally test end-to-end with an actual Azure OpenAI deployment using a custom name (e.g. gpt5nano) and a response_model to confirm structured output works
  • Confirm existing test_azure_deepseek_model_support and test_azure_mistral_and_other_models still correctly identify non-OpenAI models as is_openai_model=False (they pass in CI, but worth a glance)

Notes

  • All 60 existing Azure tests pass, plus 7 new tests
  • Requested by: João
  • Link to Devin run

Fixes #4478. The is_openai_model flag was only set based on model name
prefixes (gpt-, o1-, text-), but Azure deployment names can be anything
(e.g. gpt5nano). Now is_openai_model is also True when the endpoint is
an Azure OpenAI endpoint (openai.azure.com/openai/deployments/), ensuring
response_model, tool calling, and structured output work regardless of
the deployment name.

Co-Authored-By: João <joao@crewai.com>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] LLM call does not adhere to pydantic response_model fails for "gpt5nano"

0 participants