Skip to content

[flink] Include lake reader in source metrics and correct max timestamp lag semantics#3356

Open
zuston wants to merge 1 commit into
apache:mainfrom
zuston:lakefetch
Open

[flink] Include lake reader in source metrics and correct max timestamp lag semantics#3356
zuston wants to merge 1 commit into
apache:mainfrom
zuston:lakefetch

Conversation

@zuston
Copy link
Copy Markdown
Member

@zuston zuston commented May 21, 2026

Purpose

Linked issue: close #3304

Brief change log

  1. Add the lake reader into the source metrics, this is necessary for the long-term backfill when using the fluss multi-layer log table as the source
  2. Correct the max timestamp log semantics on subscribing the multi buckets or partitions. that is mentioned in [flink] currentFetchEventTimeLag is misleading when subscribing multiple partitions/buckets with uneven lag #3304
  3. Make these metrics collecting logic into one place in FlinkRecordsWithSplitIds

Tests

API and Format

Documentation

@zuston
Copy link
Copy Markdown
Member Author

zuston commented May 21, 2026

cc @swuferhong @luoyuxia

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[flink] currentFetchEventTimeLag is misleading when subscribing multiple partitions/buckets with uneven lag

1 participant