[DYNAMO] TITO Dynamo nvext transport follow-up#1313
Draft
AmeenP wants to merge 2 commits into
Draft
Conversation
AmeenP
force-pushed
the
codex/dynamo-renderer-transport
branch
from
f6b72e9 to
f6f8224
Compare
AmeenP
force-pushed
the
codex/dynamo-tito-nvext-followup
branch
from
985963e to
0ef29e4
Compare
…okenClient
The verifiers TITO client previously only spoke vLLM's TITO surface
(/v1/chat/completions/tokens for the final POST, /tokenize for bridge
tokenization). Dynamo bis/dynamo-rl serves neither route, so multi-turn
TITO against Dynamo silently degraded to MITO every turn-2+ via the
existing fallback path.
This commit teaches the TITO client to read ClientConfig.renderer_transport
(same field RendererClient consults) and route accordingly:
- prime_vllm_generate (default): unchanged - posts to
/v1/chat/completions/tokens and uses /tokenize over HTTP.
- dynamo_chat_nvext: bridge tokenize runs locally via the renderers
package (zero RTTs); final POST goes to /v1/chat/completions with
placeholder messages + nvext.token_data carrying the stitched
prompt_ids + explicit stop_token_ids from renderer.get_stop_token_ids().
Wire shape matches what RendererClient already produces for the
same transport, so a Dynamo deployment validated against renderer
mode automatically accepts TITO traffic too.
Adds two unit tests that assert the dynamo-transport wire shape end-to-
end via a recording client + stub renderer (no real tokenizer download).
OpenAIChatCompletionsTokenClient.get_prompt_ids' prefix-match between the prompt_messages caller-input and the trajectory step messages was asymmetric: - prompt_messages went straight through normalize_for_comparison (which picks up vf.AssistantMessage.model_dump's exhaustive view, including thinking_blocks=None and other defaulted fields). - step_messages went through to_native_prompt FIRST, which produces the slimmer OpenAI-format dict that omits thinking_blocks entirely. The two normalized forms then never compared equal whenever the caller handed the client Pydantic vf.Message types -- the form MultiTurnEnv produces after maybe_normalize_messages -- so the prefix match always returned None and TITO silently fell back to MITO every turn-2+. Probe-3 and the upstream test suite both used raw dict input, so the asymmetry only showed up under real orchestrator rollouts. Fix: drop None-valued keys in normalize_for_comparison. Both sides land on the same shape regardless of whether they came in as Pydantic or as plain OpenAI dicts. Validated end-to-end against bis-dev/5/always-continue-tito (multi-turn TITO + Dynamo bis/dynamo-rl smoke): 348 /v1/chat/completions, 21 SIDECAR-SKIP-TOKENIZE markers, 0 fall- back warnings. Same SIDECAR token-prefix appears across turns, confirming the engine reuses prior-turn ids verbatim. The existing 6 unit tests (4 vanilla TITO + 2 dynamo_chat_nvext) all still pass; their dict-shaped input always normalized to the same shape on both sides, so the symmetric drop-None doesn't change them.
AmeenP
force-pushed
the
codex/dynamo-tito-nvext-followup
branch
from
0ef29e4 to
4bb3da6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the missing TITO-side Dynamo nvext transport support from Biswa's tested verifiers branch on top of #1287.
OpenAIChatCompletionsTokenClientto userenderer_transport="dynamo_chat_nvext"/chat/completionswithnvext.token_datavf.Messageinputrenderers@7ca1ab3pin from Add Dynamo renderer transport #1287Context
This pairs with prime-rl #2446, which sets
renderer_transport="dynamo_chat_nvext"for TITO clients whenclient.backend = "dynamo".Validation
uv lock --checkuv run pytest tests/test_openai_chat_completions_token_client.py tests/test_renderer_client.pyuv run ruff check pyproject.toml verifiers/types.py verifiers/clients/renderer_client.py verifiers/clients/openai_chat_completions_token_client.py tests/test_renderer_client.py tests/test_openai_chat_completions_token_client.pyuv run ruff format --check pyproject.toml verifiers/types.py verifiers/clients/renderer_client.py verifiers/clients/openai_chat_completions_token_client.py tests/test_renderer_client.py tests/test_openai_chat_completions_token_client.py