Skip to content

WAYANG-678: Implement cardinality tracking in DataStreamChannel#729

Open
Harshakamalur wants to merge 1 commit intoapache:mainfrom
Harshakamalur:fix-datastreamchannel-cardinality
Open

WAYANG-678: Implement cardinality tracking in DataStreamChannel#729
Harshakamalur wants to merge 1 commit intoapache:mainfrom
Harshakamalur:fix-datastreamchannel-cardinality

Conversation

@Harshakamalur
Copy link

Fixes #678.

Implements cardinality tracking in DataStreamChannel.Instance using AtomicLong, following the same pattern as RddChannel in the Spark platform.

Changes:

  • Replaces the always-zero size field with an AtomicLong counter
  • Counter increments via a filter on the DataStream when instrumentation is enabled
  • Counter is read in both getMeasuredCardinality() and doDispose()
  • Mirrors the established pattern from RddChannel in wayang-spark

All 25 existing Flink tests pass.

@Harshakamalur
Copy link
Author

Hi @apache/wayang-committers, I've now submitted two PRs: this one for the README links and #728 for the TensorFlow test fixes.

Could you please assign both to me and approve the CI workflows so I can ensure all tests pass? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

this.size is currently always 0

1 participant