[Backport 7.78.x] kafka_actions: Return early when end of partition is reached#23005
[Backport 7.78.x] kafka_actions: Return early when end of partition is reached#23005dd-octo-sts[bot] wants to merge 1 commit into7.78.xfrom
Conversation
* Return early from Kafka consume_messages when partition end is reached Instead of waiting until the global timeout, break immediately when poll() returns None or when all assigned partitions report EOF. This makes the read_messages action fast when there are no more messages to consume. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add changelog entry for kafka_actions early return fix Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Simplify early return tests to 2 concise tests Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Remove enable.partition.eof and EOF tracking — just break on None Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> (cherry picked from commit 50338a5)
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c3638d4e34
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if msg is None: | ||
| continue | ||
| self.log.debug("Poll returned None (no more messages available), stopping consumption") | ||
| break |
There was a problem hiding this comment.
poll() is idle
In KafkaActionsClient.consume_messages, Consumer.poll() returning None only means the 1-second poll_timeout elapsed without a record, not that the topic is exhausted. _action_read_messages still passes the user-configured timeout_ms into this generator, so on sparse topics or producers slower than 1 msg/s this change exits after the first idle second instead of waiting for the remaining timeout and can miss messages that arrive later within the requested window.
Useful? React with 👍 / 👎.
Codecov Report❌ Patch coverage is Additional details and impacted files🚀 New features to boost your workflow:
|
3 participants
Backport 50338a5 from #23004.
Summary
consumer.poll()returnsNone(no more messages available) instead of continuing to poll until the global timeout_PARTITION_EOFcheck to useKafkaErrorinstead ofKafkaException(the constant lives onKafkaError)🤖 Generated with Claude Code