What is the difference between s2.5 and s2?

[huaiyizhao]

Hello, thank you for the great work!
I find s2.5 has a huge performance upgrade from s2 (even among the sota in OSWorld to date), but I cannot find the blog or any technical details related to s2.5. May I ask where the performance gains come from? Any documents I can refer to?

[chalo2000]

Thank you!

The only current reference we currently have for S2.5 is an X post; however, you can refer to our Agent S3 paper for details and we will update the paper to make this clearer.

In Table 2, we describe Agent S2 (no hier.) which is Agent S2.5. Section 3.3 describes the Flat Policy which is essentially Agent S2.5 (removing hierarchical planning from Agent S2). If you would like to run Agent S2.5, you can run disable Agent S3's coding actions and run a single rollout.

[huaiyizhao]

Thank for the reply! Simply put, Agent S3 = Agent S2.5 + coding + bBoN to me

[Richard-Simular]

@huaiyizhao Thanks for the issue!
To clarify, Agent S3 = Agent S2.5 + coding, bBoN is bBoN Judge + Multiple Agent S3 runs

This is the performance chart of Agent S2, S2.5, and S3

[huaiyizhao]

Hi, looking at the readme, it seems I still need a grounding model for agent s3

Is the flat policy still requires 2 seperate model?

[Richard-Simular]

@huaiyizhao
Agent S3 requires a grounding model (We keep the mixture of grounding from S2).
If you want to stick with one model I think you can reuse same model parameter for both worker and grounding expert

[huaiyizhao]

Do you compare the performance w./w.o. the grounding model? In my test, the latest multimodal model like Qwen3-VL can perform planning, executing and grounding on its own at the same time.

[Richard-Simular]

We didn't explicitly compare but tried some run of gemini3.0 pro as both grounder and planner. So far the results were not as good as having UI-Tars as a dedicated grounding expert