Skip to content

fix(dav): finalize upload metadata before post-write hooks while preserving shared hook locks#60453

Open
joshtrichards wants to merge 4 commits into
masterfrom
jtr/fix-dav-put-bookkeeping-consistency
Open

fix(dav): finalize upload metadata before post-write hooks while preserving shared hook locks#60453
joshtrichards wants to merge 4 commits into
masterfrom
jtr/fix-dav-put-bookkeeping-consistency

Conversation

@joshtrichards
Copy link
Copy Markdown
Member

@joshtrichards joshtrichards commented May 15, 2026

Summary

Tighten the final upload/publish phase in apps/dav/lib/Connector/Sabre/File.php so post-write hooks observe a fully finalized file state, while preserving the historical shared-lock behavior during hook execution.

What changed

  • extract the post-move finalization logic into finalizeUpload()
  • keep the exclusive lock through final upload bookkeeping, including:
    • updater/cache reconciliation
    • mtime / creation time / upload time persistence
    • checksum persistence
    • info refresh
  • downgrade back to a shared lock before emitting post-write hooks
  • preserve legacy hook behavior where post_write observers can still access the file during the callback
  • reduce the number of intermediate observable states during final upload publication

Why

This upload path intentionally bypasses the normal view-layer publish path and then reconciles cache/bookkeeping state manually.

Previously, the code downgraded the lock to shared too early, before all metadata and checksum updates had been applied. That created a window where the file could be visible at the final path while parts of the visibility/bookkeeping state were still catching up.

I believe this is a likely cause of transient timing-related failures, such as the FilesDrop CI failures that intermittently report the final path as visible while associated state is still incomplete:

{"reqId":"ExN63msWomhYw6LqRBee","level":3,"time":"2026-05-15T00:12:28+00:00","remoteAddr":"::1","user":"--","app":"no app in context","method":"PUT","url":"/public.php/dav/files/jHybAbajiZcXgfy/folder","scriptName":"/public.php","message":"Cannot set extra headers for non-existing file 'files/jHybAbajiZcXgfy/Alice/folder (2)'","userAgent":"GuzzleHttp/7","version":"34.0.0.6","data":[]}
{"reqId":"2pwd9QrZNEDnza5kbPnv","level":3,"time":"2026-05-15T00:12:28+00:00","remoteAddr":"::1","user":"user0","app":"webdav","method":"GET","url":"/remote.php/webdav/drop/Alice/folder/a.txt","scriptName":"/remote.php","message":"Could not open file: drop/Alice/folder/a.txt (214), file does seem to exist","userAgent":"GuzzleHttp/7","version":"34.0.0.6",[...]

Cc: @icewind1991 because you last adjusted the logging scenarios for this in #56166 - so may be related to some other work you were doing / problem you encountered.

At the same time, the DAV upload path has a long-standing expectation that hooks — especially post_write — execute while the file is only shared-locked, so hook consumers can still access the file during the callback.

This change keeps the consistency improvements from moving finalization earlier, while preserving the older hook-time locking behavior.

Consistency with chunked uploads

This also makes the direct DAV PUT path more consistent with ChunkingV2Plugin: do the critical finalization work under exclusive lock, then downgrade to shared before emitting post-write hooks. That preserves hook accessibility while still ensuring hooks see a more fully finalized state than on current master.

Locking / hook semantics after this change

The intended ordering is:

  1. acquire shared lock before PUT
  2. upgrade to exclusive for the final publish/write step
  3. finalize cache + metadata + checksum + refreshed info under exclusive lock
  4. downgrade to shared
  5. emit post-write hooks
  6. release shared lock after PUT completes

This means:

  • post_write observers should see finalized metadata/state
  • hook consumers can still access the file during the callback
  • the inconsistent “published but not fully finalized” window is narrowed significantly

Scope / non-goals

This change does not alter:

  • the part-file upload strategy
  • cross-storage final move behavior
  • the stream write path
  • pre-hook behavior
  • general filesystem hook semantics outside this DAV upload finalization path

It only tightens ordering during the final publish/finalization phase while keeping shared-lock hook compatibility.

Testing / review focus

Please review especially for:

  • lock ordering during the final publish step
  • checksum persistence ordering relative to post hooks
  • preservation of shared-lock execution during post_write
  • any assumptions around batching final metadata updates through putFileInfo()

Historical context

Shared-lock hook execution during DAV PUT was intentionally introduced years ago so post-write hook consumers could access the file during callback execution. This change tries to preserve that compatibility while still making post-hook state more consistent.

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

Signed-off-by: Josh <josh.t.richards@gmail.com>
@joshtrichards
Copy link
Copy Markdown
Member Author

/backport to stable33

@joshtrichards joshtrichards added this to the Nextcloud 34 milestone May 15, 2026
@joshtrichards joshtrichards marked this pull request as ready for review May 15, 2026 14:13
@joshtrichards joshtrichards requested a review from a team as a code owner May 15, 2026 14:13
@joshtrichards joshtrichards requested review from ArtificialOwl, CarlSchwan, icewind1991 and leftybournes and removed request for a team May 15, 2026 14:13
@joshtrichards joshtrichards added 3. to review Waiting for reviews feature: filesystem and removed 2. developing Work in progress labels May 15, 2026
…mproving consistency

Signed-off-by: Josh <josh.t.richards@gmail.com>
Signed-off-by: Josh <josh.t.richards@gmail.com>
@joshtrichards joshtrichards changed the title fix(dav): finalize upload bookkeeping before downgrading lock fix(dav): finalize upload bookkeeping before post-write hooks May 17, 2026
@joshtrichards joshtrichards changed the title fix(dav): finalize upload bookkeeping before post-write hooks fix(dav): finalize upload metadata before post-write hooks while preserving shared hook locks May 17, 2026
At least where likely to be needed. Also fixed object type.

Signed-off-by: Josh <josh.t.richards@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant