Refactor loss ctx

[jayhenry]

Summary

This PR refactors the loss subsystem to standardize how loss contexts are constructed, sequence-parallelism (SP) is handled, and batches are built.
Now Trainer/Worker only interact with LossConfig and LossContext objects.
It introduces unified builders across losses, removes legacy input-item types.

Highlights

• Introduces BaseLossConfig.build, and BaseLossContext.build_batches. Removes *ContextInputItem, and BaseLossKwargs now include built-in to, sp_split instead.
• CE loss:
  • Adds CELossConfig.build and CELossContext.build_batches.
• RL loss:
  • Adds BaseRLLossConfig, BaseRLLossKwargs, BaseRLLossContext.
  • Refactors GRPO/Oreal to the new API;
  • BaseRLLossContext internalizes rollout importance-sampling handling.
• Worker/Trainer flow:
  • Workers and trainer now only interact with LossConfig and LossContext directly
  • batching is done via LossContext.build_batches.
  • compute_actor_logprobs and compute_ref_logprobs return List[torch.Tensor] for clearer data flow.
  • SP handling is centralized within builders for consistency.

Compatibility

• Public training CLI and typical user workflows remain unchanged; only internal wiring and APIs between workers/trainers and loss configs are updated.

Testing

• Updated unit and integration tests across engines/models/losses to the new API.
• Verified identical reward curves , correct SP behavior for RL

[HIT-cwh]

xtuner/xtuner/v1/rl/base/worker.py

Line 770 in 8f8bf16

def _train_one_step_sft(self, data_batch):

_train_one_step_sft 这里也需要对应修改

[HIT-cwh]

是不是类似 CELossContextInputItem 这样的 class 都不需要了

[jayhenry]

是不是类似 CELossContextInputItem 这样的 class 都不需要了

Yes