[Doc] Memory-efficient RL training tutorial + cross-refs

[vmoens]

Summary

Ties together the three recently-merged memory-efficiency PRs into a
single story:

• compact_obs collector flag ([Performance] Add compact_obs flag to DataCollector #3742)
• NextStateReconstructor RB transform ([Feature] NextStateReconstructor RB transform #3743)
• NaN-safe value-estimator forward ([BugFix] Sanitize NaN ('next', obs) in value-estimator forward #3744)

Two parts:

1. Runnable Sphinx-gallery tutorial at
tutorials/sphinx-tutorials/memory_efficient_rl.py. Sections:

• Where the observation memory goes (concrete td.bytes() numbers)
• Why ("next", obs) is kept by default — bootstrap target at
  trajectory ends, MultiStepTransform n-step fallback
• Knob 1 — SyncDataCollector(compact_obs=True)
• Knob 2 — NextStateReconstructor with the traj_id + done contract
• Knob 2.5 — value-estimator NaN safety
  (_sanitize_next_obs_nan), GAE finite everywhere
• When not to take this path — MultiStepTransform incompatibility,
  the V(obs[t]) ≈ V(real_next_obs) approximation at truncated steps,
  and how shifted=True interacts
• Knob 3 — LazyMemmapStorage for buffers ≥ VRAM
• Knob 4 — SliceSampler + the new "scan" / "triton"
  recurrent backends for padding-free sequence training
• End-to-end pipeline snippet
• Conclusion + Further reading

Runs end-to-end on CPU (CartPole-v1, 200 frames; <2s wall) and reports
the byte-level savings concretely from td.bytes().

2. Docstring cross-references so a reader landing on any of the
three new APIs finds the other two:

• Collector(compact_obs=…) (and the multi-process collectors):
  pointers to NextStateReconstructor, the value-estimator
  sanitizer, the MultiStepTransform incompatibility note, and the
  new tutorial.
• NextStateReconstructor: .. seealso:: block covering
  compact_obs, the sanitizer, MultiStepTransform, and the
  tutorial.
• ValueEstimatorBase._sanitize_next_obs_nan: .. seealso::
  block to compact_obs, NextStateReconstructor, and the
  tutorial.

docs/source/index.rst registers the new tutorial under "Basics".

Test plan

• Tutorial runs end-to-end with concrete output (memory savings
  reported, NaN at slice boundaries confirmed to coincide with
  trajectory boundaries, GAE advantage finite everywhere, memmap
  roundtrip works).
• All cross-references resolve to existing public symbols
  (verified by reading the rendered class docstrings via
  Collector.__doc__ etc.).

🤖 Generated with Claude Code

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3745

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Run pull request jobs on OSDC runners in shadow mode

❌ 2 New Failures, 2 Pending

As of commit bf4a3f6 with merge base cc31dc3 ():

NEW FAILURES - The following jobs have failed:

• Lint / python-source-and-configs / linux-job (gh)
• Unit-tests on Linux / tests-cpu (3.13) / linux-job (gh)
  test/objectives/test_ddpg.py::TestDDPG::test_ddpg_prioritized_weights

This comment was automatically generated by Dr. CI and updates every 15 minutes.