Is Motion Clustering and Specialist-to-Generalist mentioned in the paper implemented in AnyTracker?

[Skevinci]

Hi,

After successfully reproducing the code for AnyTracker and reviewing the codebase, I noticed that the current implementation appears to use a simple MLP actor–critic architecture trained with PPO on multiple motions, along with domain randomization during training.

I wanted to ask whether the Motion Clustering and Specialist-to-Generalist methods described in the paper are actually implemented in AnyTracker. Additionally, the paper seems to suggest that domain randomization was not included in AnyTracker, which seems to differ from what I observed. In my reproduction, the robot also struggles with contact-rich actions such as fallAndGetup, which the paper reported as being successfully performed.

Please feel free to correct me if I’ve misunderstood any part of the codebase or the paper’s implementation details.

Thank you so much.

[STONEKONG]

I completely agree with your observations. I ended up running all of my tests purely in simulation. Have you by any chance deployed the released checkpoint on a real robot? If so, which platform and how did you handle the sim-to-real gap? I would love to hear your hardware-side findings.

[zzk273]

The current version of OpenTrack is not exactly the implementation of Any2Track. It's just the codebase of Any2Track. Nevertheless, it should be fully capable of handling the fall and get-up motions. Could you let me know which specific motion clip encountered the issue?

[Skevinci]

If I train it on all motion clips, the universal policy fail to handle the fall and get-up motion. But if I only train one policy for a single fall and get-up motion, it works.