Skip to content

Performance: stream-head background updates schedule a Task.Run per commit under write load #78

@AGiorgetti

Description

@AGiorgetti

Summary

When snapshot support is enabled and background stream-head updates are on, each successful commit calls UpdateStreamHeadInBackgroundThread(...) via Task.Run. This schedules a new thread-pool work item per commit, adding task-scheduling overhead and GC pressure at high write rates.

Why this matters

  • Task-scheduling overhead accumulates at high write rates.
  • Fire-and-forget work can add GC pressure and increase tail latency.
  • Benchmarks show a 2.5× write-time increase with background updates enabled vs disabled.

Baseline (net10, 2026-05-20)

Benchmark Mean
WriteToStream(CommitsToWrite=1000, DisableSnapshotSupport=False, PersistStreamHeadsOnBackgroundThread=True) 2,631.6 ms
WriteToStream(CommitsToWrite=1000, DisableSnapshotSupport=False, PersistStreamHeadsOnBackgroundThread=False) 1,059.4 ms

Delta: +148% overhead with background updates enabled.

Proposed investigation

  • Determine whether the 2.5× overhead is from task-scheduling cost or actual MongoDB write contention from background updates.
  • Evaluate batching or debouncing stream-head updates rather than scheduling one Task.Run per commit.
  • Consider a bounded channel / dedicated background worker to coalesce stream-head updates.

Reference

See docs/Performance-Investigation.md → Finding #6 for full context.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions