Skip to content

dispatcher,event,cloudstorage: fix enqueue callback semantics and race #4269

@3AceShowHand

Description

@3AceShowHand

Problem

The current DML two-stage ack implementation in storage sink has three correctness/semantics gaps:

  1. PostEnqueue is triggered too early in cloud storage sink.

    • It is currently fired in dmlWriters.AddDMLEvent right after pushing into msgCh.
    • This means "enqueue" only guarantees entering the first unbounded internal queue, not acceptance into the downstream write pipeline.
  2. DMLEvent.PostEnqueue is not concurrency-safe.

    • PostEnqueue can be called both from enqueue path and via PostFlush fallback.
    • The guard is a plain boolean, so concurrent calls can cause a data race and potentially double invocation.
  3. FilterDMLEvent may lose enqueue callbacks.

    • When a filtered event is rebuilt as a new DMLEvent, PostTxnEnqueued is not migrated.
    • This can silently drop enqueue-stage callbacks on filtered paths.

Expected behavior

  • Enqueue-stage callback should represent acceptance into sink's write pipeline (after encoding/defragment dispatch), not just source queue insertion.
  • PostEnqueue should be exactly-once and race-free under concurrent calls.
  • Filtering/rebuilding should preserve callback semantics (PostTxnEnqueued and related state).

Related PR

Metadata

Metadata

Assignees

Labels

type/enhancementThe issue or PR belongs to an enhancement.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions