Add full path entries for vertex for newer gemini models#726
Add full path entries for vertex for newer gemini models#726Everett Smith (evrtt) wants to merge 2 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2cb5a34199
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "Codex (@codex) review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "Codex (@codex) address that feedback".
| "supported_regions": [ | ||
| "global" | ||
| ], |
There was a problem hiding this comment.
Route the global-only Vertex model through global
For Vertex requests without an explicit secret metadata location, this new full-path model will still be sent to the default us-central1 location because vertexEndpointInfo only consults modelSpec.locations when choosing the Vertex location (packages/proxy/src/proxy.ts:2633-2640), not supported_regions. Since this entry advertises only global, calls to publishers/google/models/gemini-3.1-flash-image-preview can still 404 in the default proxy path unless the user happens to configure the secret location as global; mirror the existing global Vertex Gemini entries by setting locations: ["global"] for routing.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
resolved w/ commit 88d6377
User facing issue
SDK invoke() prompt throws 404 error w/ some newer gemini models—
gemini-3.5-flash,gemini-3.1-flash-lite,gemini-3.1-flash-image-preview,gemini-2.5-flash-image—on Vertex. These prompts work from the playground.Root Cause
The short model names—e.g.
gemini-3.5-flash—are mapped tovertexfor these models instead of or in addition to the full publisher model name—e.g.publishers/google/models/gemini-3.5-flash. When these short name models are selected from the braintrust model registry, this causes the Proxy to construct requests for Vertex with short model names at the end of the path causing Vertex to throw a nondescript404.The prompts work in the playground because it follows a newer (gateway) code path where the full model name is constructed for the models. This doesn't happen for older (proxy) code path.
Fix
Some were already had separate full model provider name mappings for vertex. Effectively, the proposed fix is to remove vertex from the mapping on these short model entries and create or leave, full publisher model entries mappings for vertex.
Would become
This aligns with prior config for gemini models with duplicate mapping entries for google / vertex.
Also adding entries in model_list for the two full path models that are now missing,
publishers/google/models/gemini-3.1-flash-image-previewandpublishers/google/models/gemini-2.5-flash-imagemirroring the short model name entries.Testing
No testing changes needed, tested locally and prompts with all four models now work w/ invoke() and from playground