fix: prevent duplicate lineage cleanup and remove blocking refresh in PATCH /api/v1/tables/{id}

[kojikokojiko]

Fixes #26674

Root Cause

Two issues caused PATCH /api/v1/tables/{id} to be extremely slow (49–99s) when a database service had many lineage entries:

Issue 1: Duplicate deferred operations during session consolidation

When the same user edits the same entity within 10 minutes, session change consolidation is triggered. In this path, updateInternal() is called multiple times:

• once in incrementalChange() for incremental diff calculation
• once or twice in revert() for consolidation
• once in the final flush phase

Each call traversed entitySpecificUpdate() → updateColumns() → handleColumnLineageUpdates() → deferReactOperation(), registering deleteColumnsInUpstreamLineage as a deferred operation 2–4 times per PATCH request.

Fix: Add a deferOpsEnabled flag to EntityUpdater that is set to false during intermediate calculations (incrementalChange and revert). Deferred operations are only registered during the final updateInternal call.

Issue 2: Blocking refresh in updateByQuery

deleteColumnsInUpstreamLineage and updateColumnsInUpstreamLineage used Refresh.True in updateByQuery, which blocks until all shards have refreshed after updating potentially thousands of documents. With 13,275 matching documents, this caused ~90s of blocking per call.

Fix: Remove the refresh parameter, allowing ES/OpenSearch to refresh on its normal schedule (~1s), which is sufficient for lineage cleanup.

Verification

Confirmed with local testing:

Duplicate call fix: Server logs showed deleteColumnsInUpstreamLineage executing 3 times per PATCH with consolidation before the fix, and 1 time after.

Refresh fix: Direct ES measurement with 1,000 documents:

	Time
refresh=true (before)	0.245s
no refresh (after)	0.124s
Improvement	~50% reduction

Combined effect (3x duplicate calls × blocking refresh): expected >80% reduction in PATCH latency for tables with many lineage entries.

Test plan

• Run ingestion workflow on a database service with 700+ lineage entries and verify PATCH /api/v1/tables/{id} completes in reasonable time
• Verify server logs show deleteColumnsInUpstreamLineage appears only once per PATCH request even with session consolidation active
• Verify lineage data is eventually consistent in search after column deletion (within ~1s)

🤖 Generated with Claude Code

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[github-actions]

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

[gitar-bot]

Code Review ✅ Approved

Fixes duplicate lineage cleanup and removes blocking refresh in PATCH /api/v1/tables/{id} to improve performance. No issues found.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply	Compact
gitar auto-apply:on	gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}