[Tracking]: Beam 3.0.0 - Milestone 1 Key Features

[Abacn]

What needs to happen?

This umbrella issue tracks key features as part of Beam 3.0.0 milestones

ref: Beam Summit 2025 keynote - https://beamsummit.org/slides/2025/the-future-of-beam.pdf

3 Pillars of Beam 3.0

Access for all

Task	sub task	timeline (proposed)	Notes
Expansive IO Ecosystem
Turnkey transforms
Beam YAML (documentation)			First stable release 2.52.0
Structured Data (and transforms)
	Schema support improvement (#34672)
CDC / Lakehouse

"Just works" ops

Task	sub task	timeline (proposed)	Notes
Managed Transforms
	Introduced Managed Transform API (#30830) and Iceberg Managed IO	2.56.0	Dataflow managed I/O support since Beam 2.58.0
	Kafka Managed IO (#31172)	2.57.0
	BigQuery Managed IO (#31486)	2.61.0
	Jdbc managed IO (#36034)	2.68.0 (postgres)
Fully Featured Local Experience
	Prism Runner (#29650) as default direct runner (#34549)	2.50.0 (Go); 2.68.0 (Python)
Whole app tracing
	Opentelemetry support (#33176)	tbd
	Lineage support	stable as of Beam 2.66.0 (#34342)	OpenLineage support as stretching goal Open (#33981)

First class ML

Task	sub task	timeline (proposed)	Notes
Model Serving
	LLM remote inference: Gemini (#35036) and OpenAI (#36081)
	Java Remote RunInference (#36253)
	RemoteModelHandler API (#34379) and its implementations	2.65.0
Feature Generation
	Enrichment support for: CloudSQL (#30773), Milvus (#36176), and Tecton (#36062)
	Sinks for: Milvus (#35944), and Tecton (#36062)
Embedding Generation
	Multimodal Embedding support: #34236
	Embedding ingestion support for: MySQL (#35393), Postgres (#35215), and AlloyDB (#34156)
Anomaly Detection (#35706)			Stable as of Beam 2.65.0
New Capabilities
	ML-aware API rate limiter

---

For the changes on existing features/API change for Beam 3.0.0, see SDK trackers:

• [Tracking]: Beam 3.0.0 Milestone 1 Java SDK breaking changes #35836 Java SDK breaking changes
• [Tracking]: Beam 3.0.0 Milestone 1 Python SDK breaking changes #36101 Python SDK breaking changes
• [Tracking]: Beam 3.0.0 Milestone 1 Go SDK breaking changes #36102 Go SDK breaking changes

[damccorm]

Issues/PRs that stand out to me for First class ML:

Model Serving

• Java Remote RunInference - @jrmccluskey do we have an issue for this?
• Support for LLM remote inference: Gemini and OpenAI

Feature Generation

• Enrichment support for: CloudSQL, Milvus, and Tecton
• Sinks for: Milvus, and Tecton

Embedding Generation

• Multimodal Embedding support
• Embedding ingestion support for: MySQL, Postgres, and AlloyDB

Anomaly Detection

• Support added for anomaly detection transform

@claudevdm @jrmccluskey @shunping @tvalentyn could you please add anything that I am missing?

[jrmccluskey]

Tracking issue for Java Remote Inference filed at #36253

Would we also want to add the model evaluation work here?

[damccorm]

Yes, that's a good idea

[liferoad]

ML-aware API rate limiter on the SDK side could be another work. cc @tarun-google

[vampire-hunter-lincoln]

Hi Beam Team,

Do we have any estimate timeline for Opentelemetry support? I do see a few related PR being merged recently. But I am not sure when Opentelemetry support will be ready.