Skip to content

refactor(header): replace IsEmptyBlock with fast IsZero short-circuit#2547

Merged
ValentaTomas merged 2 commits intomainfrom
refactor/header-iszero-fast-path
May 4, 2026
Merged

refactor(header): replace IsEmptyBlock with fast IsZero short-circuit#2547
ValentaTomas merged 2 commits intomainfrom
refactor/header-iszero-fast-path

Conversation

@ValentaTomas
Copy link
Copy Markdown
Member

@ValentaTomas ValentaTomas commented May 3, 2026

IsEmptyBlock was the per-block hot path of DiffMetadataBuilder.Process — one call per block during the rootfs scan in DirectProvider.exportToDiff. It paid a full SIMD memcmp (4 KiB or, on hugepages, 2 MiB) before it could short-circuit, and required two static zero buffers (EmptyBlock, EmptyHugePage) to compare against.

Replace it with a small IsZero helper that uses QEMU's buffer_is_zero trick: sample three bytes (first, last, middle) so most non-zero blocks reject from a single cache line. The fallback uses bytes.Equal on b shifted by one (b[1:] == b[:n-1]), which dispatches to the runtime's SIMD memequal on amd64/arm64 — same speed as the old comparison against a static buffer when the short-circuit doesn't fire.

DiffMetadataBuilder.Process calls IsZero directly (the size validation IsEmptyBlock did was unreachable in practice). EmptyBlock is removed; EmptyHugePage stays — build.go uses it as the literal zero buffer for uuid.Nil reads in Build.ReadAt.

Pre-factored from #2546 so it can land on its own.

@cursor
Copy link
Copy Markdown

cursor Bot commented May 3, 2026

PR Summary

Medium Risk
Touches the hot-path logic that decides whether a block is treated as empty vs dirty; any false positives would silently drop data from diffs. The new heuristic is simple but should be validated across all block sizes and architectures to avoid edge-case misclassification.

Overview
Replaces per-block zero detection in DiffMetadataBuilder.Process by removing IsEmptyBlock and introducing a faster IsZero check that short-circuits on sampled bytes before a SIMD-backed equality fallback, and adds unit tests covering sampled and buried non-zero cases.

Reviewed by Cursor Bugbot for commit 88dfb98. Bugbot is set up for automated code reviews on this repo. Configure here.

Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small, well-tested refactor of a leaf hot-path utility — only finding is a doc-comment nit, safe to land.

Extended reasoning...

Overview

This PR refactors header.IsEmptyBlock (per-block hot path of the rootfs diff scan) in packages/shared/pkg/storage/header/diff.go plus adds a new diff_test.go. The substantive changes are: (1) a new IsZero helper that samples 3 bytes (first/last/middle) before falling back to a SIMD-accelerated self-equality compare, (2) deletion of the now-unused EmptyBlock global, (3) explicit length validation in IsEmptyBlock, (4) updated doc comments. About 40 lines net added, all in a self-contained leaf utility.

Security risks

None. This is pure CPU-side logic over already-loaded buffers — no auth, no I/O, no untrusted input parsing. The new length check actually tightens the API surface (callers can no longer silently pass a wrong-shape buffer). The deletion of EmptyBlock is verified safe: grep shows no remaining references after the patch.

Level of scrutiny

Low. Tight scope, behavior-preserving by intent, and the correctness argument for IsZero is straightforward to verify: for n>3, the 3-byte sample forces b[0]==0, and bytes.Equal(b[:n-1], b[1:]) then transitively requires all adjacent bytes to be equal, so all bytes must be zero. For n≤3, the sample alone covers every index. The new tests exercise nil, len≤3, full-block zero, and non-zero bytes hidden between sample positions. Length-mismatch and unsupported-block-size paths are also tested.

Other factors

The single bug finding is a doc-comment nit: the new comment for EmptyHugePage references Build.ReadAt, but the actual consumer is File.Slice at build.go:107 (verified via grep — no Build.ReadAt exists in this repo). Doc-only, no runtime impact, doesn't justify blocking. The PR description carries the same minor inaccuracy. Otherwise the change is mechanical and well-motivated.

Comment thread packages/shared/pkg/storage/header/diff.go Outdated
@ValentaTomas ValentaTomas force-pushed the refactor/header-iszero-fast-path branch from a44d240 to 11a2351 Compare May 3, 2026 22:06
@ValentaTomas ValentaTomas changed the title refactor(header): fast IsZero short-circuit for IsEmptyBlock refactor(header): replace IsEmptyBlock with fast IsZero short-circuit May 3, 2026
@ValentaTomas ValentaTomas force-pushed the refactor/header-iszero-fast-path branch 2 times, most recently from 15ccb7d to eccf472 Compare May 3, 2026 22:26
@ValentaTomas
Copy link
Copy Markdown
Member Author

Trimmed IsZero doc to one paragraph and dropped the trivial test cases — kept just the cases that exercise the 3-byte sample, the SIMD fallback path, and the byte-just-before-the-last-sample edge case. Also fixed the EmptyHugePage callsite reference to (*build.File).Slice (was Build.ReadAt per Claude review).

@ValentaTomas ValentaTomas force-pushed the refactor/header-iszero-fast-path branch 2 times, most recently from b00cc57 to 42bd9c7 Compare May 3, 2026 22:32
IsEmptyBlock is the per-block hot path of DiffMetadataBuilder.Process,
called once per snapshot block during DirectProvider.exportToDiff. Two
small wins:

- Always paid a full SIMD memcmp (4 KiB or 2 MiB on hugepages) before it
  could short-circuit. Borrow QEMU's buffer_is_zero trick: sample three
  bytes (first, last, middle) so most non-zero blocks reject from a
  single cache line.
- Drop the unused EmptyBlock global; only the header itself referenced
  it. EmptyHugePage stays — build.go uses it as a literal zero buffer
  for nil-build reads.

The fallback uses bytes.Equal on b shifted by one (b[1:] == b[:n-1])
which dispatches to the runtime's SIMD-accelerated memequal on
amd64/arm64, same speed as the old comparison against a static buffer.

Also adds a length check to IsEmptyBlock so a too-small/too-large buffer
returns an explicit error instead of silently reading zero.
@ValentaTomas ValentaTomas force-pushed the refactor/header-iszero-fast-path branch from 42bd9c7 to 7743341 Compare May 3, 2026 22:33
@ValentaTomas ValentaTomas enabled auto-merge (squash) May 3, 2026 22:33
ValentaTomas added a commit that referenced this pull request May 3, 2026
Strip verbose doc comments and inline narration across the NBD/cache
diff-shrinking change. Drop redundant dispatch tests (kept the two that
actually exercise WRITE_ZEROES routing) and simplify mockProvider to
the surface the remaining tests use. Sync diff.go/diff_test.go/metadata.go
with #2547 so the duplicated content matches once that PR rebases in.

No behaviour changes.
"iff" is logic shorthand, not a typo, but it keeps tripping readers in a
plain-English code comment. Replace with "exactly when".
@ValentaTomas ValentaTomas merged commit 3b3f8d9 into main May 4, 2026
45 checks passed
@ValentaTomas ValentaTomas deleted the refactor/header-iszero-fast-path branch May 4, 2026 08:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants