Skip to content

fix: add GenerationChangedPredicate to stop reconcile churn from status patches#33

Open
zachsmith1 wants to merge 3 commits intomainfrom
fix/reduce-reconcile-churn
Open

fix: add GenerationChangedPredicate to stop reconcile churn from status patches#33
zachsmith1 wants to merge 3 commits intomainfrom
fix/reduce-reconcile-churn

Conversation

@zachsmith1
Copy link
Copy Markdown
Contributor

Summary

  • Add GenerationChangedPredicate to For() watches on all three replicator controllers and to Owns() on the zone replicator, preventing status-only updates from re-enqueuing the reconciler
  • Use explicit Requeue: true after metadata mutations (finalizer, owner ref) since those don't bump generation and the predicate now filters them
  • Rename dnszonediscovery-replicator controller to dnszonediscovery
  • Add 12 status idempotency tests proving updateStatus and ensureDownstream* produce zero writes in steady state

Context

Status patches on the upstream object were bouncing back through the For() watch and re-enqueuing the reconciler, since there was no predicate to filter out status-only updates (which don't bump metadata.generation). Every downstream status mirror caused at least one wasted reconcile cycle. On the zone replicator, Owns(&DNSRecordSet{}) without a predicate compounded this by triggering zone reconciliation on every owned recordset status mirror.

Test plan

  • All existing tests pass
  • New status_idempotency_test.go covers:
    • updateStatus no-op when upstream already matches downstream (recordset + zone)
    • nil vs empty slice handling for RecordSets, Conditions, and per-record Conditions
    • ensureDownstream* second-call idempotency (no patch on second call)
    • Full steady-state zone reconcile flow producing zero writes to upstream and downstream
    • SetStatusCondition stability when re-applied with different LastTransitionTime
  • Deploy to staging and verify reconcile logs show reduced churn

…us patches

Status patches on the upstream object were re-enqueuing the reconciler
via the For() watch because there was no predicate filtering out
status-only updates (which don't bump metadata.generation). This caused
an unnecessary reconcile cycle after every status mirror from downstream.

On the zone replicator, Owns(&DNSRecordSet{}) without a predicate also
triggered zone reconciliation whenever an owned recordset's status was
mirrored — the zone only needs to react to recordset creates, deletes,
and spec changes.

Changes:
- Add GenerationChangedPredicate to For() on all three replicator
  controllers and to Owns() on the zone replicator
- Use explicit Requeue: true after metadata mutations (finalizer, owner
  ref) since the predicate now filters those watch events
- Rename dnszonediscovery-replicator to dnszonediscovery (not a
  replicator)
- Add status idempotency tests proving updateStatus and
  ensureDownstream methods produce zero writes in steady state
…ators

The zone and recordset replicators rely on the For() watch re-enqueuing
after status patches to drive convergence — the downstream watch alone
does not reliably deliver events in time (confirmed by e2e failures).

Keep GenerationChangedPredicate only on dnszonediscovery which is
one-shot and has no downstream watch dependency. The status update
methods are proven idempotent by tests, so the re-enqueue from the
For() watch produces a read-only no-op reconcile.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant