[Feature] Adds HERReplayBuffer and HindsightStrategy to torchrl.data.

[theap06]

Description

Describe your changes in detail.

Motivation and Context

Fixes #3713

I have raised an issue to propose this change (required for new features and bug fixes)

##Summary

• Adds HERReplayBuffer — a TensorDictReplayBuffer subclass that applies goal relabeling at sample time, turning failed goal-conditioned trajectories into useful training signal (Andrychowicz et al., NeurIPS 2017)
• Adds HindsightStrategy enum with all four canonical strategies: FUTURE (recommended), FINAL, EPISODE, RANDOM
• Exports both symbols from torchrl.data and torchrl.data.replay_buffers

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3734

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 16 Awaiting Approval

As of commit 12be7a3 with merge base eb90c5d ():

AWAITING APPROVAL - The following workflows need approval before CI can run:

• Build Aarch64 Linux Wheels (gh)
• Build Linux Wheels (gh)
• Build M1 Wheels (gh)
• Build Windows Wheels (gh)
• Continuous Benchmark (PR) (gh)
• Generate documentation (gh)
• Habitat Tests on Linux (gh)
• Libs Tests on Linux (gh)
• Lint (gh)
• LLM Tests on Linux (gh)
• Push Binary Nightly (gh)
• SOTA Tests on Linux (gh)
• Tutorials Tests on Linux (gh)
• Unit-tests on Linux (gh)
• Unit-tests on Windows (gh)
• Validate Test Partitioning (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

⚠️ PR Title Label Error

PR title must start with a label prefix in brackets (e.g., [BugFix]).

Current title: Adds HERReplayBuffer and HindsightStrategy to torchrl.data.

Supported Prefixes (case-sensitive)

Your PR title must start with exactly one of these prefixes:

Prefix	Label Applied	Example
[BugFix]	BugFix	[BugFix] Fix memory leak in collector
[Feature]	Feature	[Feature] Add new optimizer
[Doc] or [Docs]	Documentation	[Doc] Update installation guide
[Refactor]	Refactoring	[Refactor] Clean up module imports
[CI]	CI	[CI] Fix workflow permissions
[Test] or [Tests]	Tests	[Tests] Add unit tests for buffer
[Environment] or [Environments]	Environments	[Environments] Add Gymnasium support
[Data]	Data	[Data] Fix replay buffer sampling
[Performance] or [Perf]	Performance	[Performance] Optimize tensor ops
[BC-Breaking]	bc breaking	[BC-Breaking] Remove deprecated API
[Deprecation]	Deprecation	[Deprecation] Mark old function
[Quality]	Quality	[Quality] Fix typos and add codespell

Note: Common variations like singular/plural are supported (e.g., [Doc] or [Docs]).

[theap06]

@vmoens Results utilizing HER. Wrote up a quick script for DDPG

[vmoens]

Amazing, I love this.
I left a couple of high level comments can you have a look?

[vmoens]

Good progress, left a few more comments here