[Feature Request]: Is KafkaIO compatible with Apache Kafka 4.0

[DuanHen]

What would you like to happen?

Hi Beam team,

I'm currently using Apache Beam 2.64.0 with kafka-clients version 3.9.0. Since Apache Kafka 4.0.0 was officially released in March 2025, I’d like to ask whether KafkaIO has been tested or confirmed to be compatible with this version.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[liferoad]

trying this with #35377

[liferoad]

This will be hard for Beam to support Kafka 4.0 soon due to https://www.automq.com/blog/apache-kafka-40-features-amp-changes#technical-requirements-and-dependencies:

Kafka 4.0 raises the minimum Java version requirements across its components:

• Kafka Clients and Kafka Streams now require Java 11
• Kafka Brokers, Connect, and Tools now require Java 17[8]

[DuanHen]

This will be hard for Beam to support Kafka 4.0 soon due to https://www.automq.com/blog/apache-kafka-40-features-amp-changes#technical-requirements-and-dependencies:

Kafka 4.0 raises the minimum Java version requirements across its components:

• Kafka Clients and Kafka Streams now require Java 11
• Kafka Brokers, Connect, and Tools now require Java 17[8]

Thank you for the clarification. I understand the current limitations regarding Kafka 4.0 support due to the updated Java requirements. I’ll continue using kafka-clients 3.9.0 for now, and once Beam supports 4.0 in the future, I’ll follow up in this issue. Thanks again for your help!

[github-actions]

This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.

[DuanHen]

Could you please let me know if there are any updates regarding this issue