Skip to content

Fix ordering of Gemini inlined batch responses#2124

Open
neo-con wants to merge 1 commit intogoogleapis:mainfrom
neo-con:fix-inlined-batch-order-1909
Open

Fix ordering of Gemini inlined batch responses#2124
neo-con wants to merge 1 commit intogoogleapis:mainfrom
neo-con:fix-inlined-batch-order-1909

Conversation

@neo-con
Copy link

@neo-con neo-con commented Mar 5, 2026

Summary

Fixes #1909.

The Gemini API can return inlinedResponses out of request order for batch jobs that use inlined_requests. This change preserves the original request order by stamping an SDK-reserved order marker on each inlined request and reordering returned inlined responses by that marker when the full marker set is present and valid.

This change keeps the behavior conservative:

  • if any response is missing the reserved order marker, preserve backend order
  • if any response has a non-numeric order marker, preserve backend order
  • caller-provided metadata cannot override the reserved SDK order marker

Changes

  • add a reserved internal metadata key for inlined request ordering
  • stamp deterministic request-order metadata onto Gemini inlined batch requests
  • reorder Gemini inlined batch responses by that metadata before returning them
  • strip the internal order marker from returned response metadata after reordering
  • add regression coverage for:
    • internal order metadata injection
    • reserved-key override behavior
    • deterministic response reordering
    • missing-metadata fallback
    • non-numeric fallback

Testing

env -u GOOGLE_API_KEY -u GEMINI_API_KEY python -m py_compile google/genai/batches.py google/genai/tests/conftest.py google/genai/tests/batches/test_create_with_inlined_requests.py
env -u GOOGLE_API_KEY -u GEMINI_API_KEY GOOGLE_GENAI_REPLAYS_DIRECTORY=/tmp/google-genai-replays pytest google/genai/tests/batches/test_create_with_inlined_requests.py --mode replay -q
env -u GOOGLE_API_KEY -u GEMINI_API_KEY GOOGLE_GENAI_REPLAYS_DIRECTORY=/tmp/google-genai-replays pytest google/genai/tests/batches/test_create_with_inlined_requests.py -k internal_order_metadata --mode replay -q

@google-cla
Copy link

google-cla bot commented Mar 5, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Batch API InlinedResponses are not returned in order

1 participant