Skip to content

[KIP-932] Support callbacks for share consumer python binding#2252

Open
Kaushik Raina (k-raina) wants to merge 7 commits into
dev_kip-932_share_consumer_commitfrom
dev_kip-932_share_consumer_existing_cb_support
Open

[KIP-932] Support callbacks for share consumer python binding#2252
Kaushik Raina (k-raina) wants to merge 7 commits into
dev_kip-932_share_consumer_commitfrom
dev_kip-932_share_consumer_existing_cb_support

Conversation

@k-raina
Copy link
Copy Markdown
Member

@k-raina Kaushik Raina (k-raina) commented May 24, 2026

Summary

  1. Enables error_cb, throttle_cb, log_cb, and oauth_cb on ShareConsumer, which dispatch through the share-consumer drain paths i.e poll, commit_sync, commit_async.
  2. Temporarily Rejects stats_cb, statistics.interval.ms, and on_commit at construction time with ValueError. Pre-filter in ShareConsumer_init scans args[0] + kwargs before common_conf_setup.
  3. Wires the share-consumer log queue onto rk_rep via rd_kafka_share_set_log_queue(rkshare, NULL) so log records reach the Python logger through the same drain as poll/commit.
  4. OAuth init contract: rd_kafka_share_sasl_background_callbacks_enable + wait_for_oauth_token_set so construction returns a ready-to-use client. wait_for_oauth_token_set refactored to no longer own destroy on failure, and each _init now invokes the correct teardown API i.e rd_kafka_destroy vs rd_kafka_share_destroy
  5. All callback trampolines now use the librdkafka-supplied rk arg in rd_kafka_yield / rd_kafka_oauthbearer_set_token calls instead of h->rk . Also error_cb additionally tolerates dispatch with no CallState and saves/restores any pending Python exception across the user callback so the constructor's original error isn't silently swallowed.
  6. New unit suite tests/test_ShareConsumer_callbacks.py covers dispatch, exception propagation, config rejection, and the CallState wrap on commit paths against an unreachable broker. Integration: throttle_cb under tests/integration/share_consumer/ and full OAUTHBEARER handshake/refresh/auth-failure under tests/integration/share_consumer_oauth/

@confluent-cla-assistant
Copy link
Copy Markdown

🎉 All Contributor License Agreements have been signed. Ready to merge.
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.

@k-raina Kaushik Raina (k-raina) changed the title [KIP-932] Support callbacks for share consumer binding [KIP-932] Support callbacks for share consumer python binding May 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant