[Feature] ToolCompose with parallel dispatch, builtin tools, legacy adapter

[vmoens]

Stack from ghstack (oldest at bottom):

• [Feature] MCP and HTTP tools, agentic tutorial, see-also pointers #3737
• -> [Feature] ToolCompose with parallel dispatch, builtin tools, legacy adapter #3736
• [Feature] Agentic toolkit foundation: protocols, parsers, sandbox, REPL #3735

Lands the headline orchestrator: ToolCompose(Compose) parses each
assistant message once, dispatches matched tools concurrently via
asyncio.gather, and injects structured results into History. ChatEnv is
unchanged; ToolCompose drops into any TransformedEnv.

Public API additions in torchrl.envs.llm.agentic:

• ToolCompose: Tools-only Compose subclass. Raises TypeError on non-Tool
  insert. Owns the parser, honors pass_state_to_tools (mirrors the legacy
  ExecuteToolsInOrder knob), supports per-tool RateLimiter, and surfaces
  three keys per step: ("agentic","any_tool_calls"),
  ("agentic","any_error"), ("agentic","stop_requested").
• DispatchResult dataclass aggregating one batch item's outcome.
• RateLimiter combining asyncio.Semaphore + token bucket.
• PythonTool (Repl-backed, state persists), ShellTool (Sandbox-backed),
  FileReadTool, StopTool. StopTool raises StopSignal which the dispatcher
  translates into the stop_requested flag.
• as_tool(transform, ...) -- legacy adapter lifting any
  ToolTransformBase-shape object (PythonInterpreter, BrowserTransform,
  MCPToolTransform, SimpleToolTransform) into a Tool. Existing user code
  drops into ToolCompose without rewriting.

Stable call_id is enforced end-to-end: parsers assign deterministic ids,
ToolCompose populates ToolContext.call_id, and parsers' render_result
echoes the same id back into the tool message injected into History.

Nested-loop safety: when ToolCompose._step is called from inside a
running event loop (e.g. Jupyter), dispatch is offloaded to a worker
thread that owns its own loop. asyncio.run is used otherwise.

Tests extend test/llm/test_llm_transforms.py with TestToolCompose,
TestPythonTool, TestShellTool, TestLegacyAdapter (19 tests):
parallel-dispatch wall-time check (3 x 500ms < 0.9s), StopTool
termination, schema validation rejection, raise-on-non-Tool insert,
duplicate-name rejection, pass_state_to_tools on/off, rate-limit
serialization, failure isolation, stable call_id round-trip, nested-loop
safety, legacy adapter end-to-end. All existing legacy
ExecuteToolsInOrder/XMLBlockParser/JSONCallParser tests still pass.

benchmarks/test_llm.py gains an "agentic-dispatch" group with parallel
n=3 / n=8 and a single-call baseline so reviewers can see parallel
dispatch flattens the wall-time curve.

Co-Authored-By: Claude Opus 4.7 noreply@anthropic.com

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3736

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 New Failures

As of commit d77e0e5 with merge base d386287 ():

NEW FAILURES - The following jobs have failed:

• Lint / lint-done (gh)
  Process completed with exit code 1.
• Lint / python-source-and-configs / linux-job (gh)
  torchrl/envs/llm/agentic/tools/builtin.py:20:1: F401 '..protocols.Tool' imported but unused
• LLM Tests on Linux / unittests-vllm (3.12, 12.9) / linux-job (gh)
  test/llm/test_wrapper.py::TestLogProbsComparison::test_log_probs_consistency[unpadded-tokens]
• Unit-tests on Linux / tests-olddeps (3.10, 11.8) / linux-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_scan_prototype
• Unit-tests on Linux / tests-optdeps (3.12, 13.0) / linux-job (gh)
  test/test_custom_envs.py::TestPendulum::test_pendulum_env[device1]
• Unit-tests on Windows / unittests-cpu (3.10, windows.4xlarge, cpu) / windows-job (gh)
  test/test_tensordictmodules.py::TestGRUModule::test_gru_scan_prototype

This comment was automatically generated by Dr. CI and updates every 15 minutes.