perf+ratio(encoder): block splitter donor-parity check vs ZSTD_splitBlock

[polaz]

Context

Donor ZSTD_splitBlock (lib/compress/zstd_compress_superblock.c) decides
whether to split a literals/sequences block at intermediate boundaries
to improve ratio. Our block_splitter::should_split uses a different
heuristic and may pick different boundaries than donor on the same input.

Goal

Build a Rust↔FFI boundary comparator that runs both block splitters on
the same input and reports:

1. For each input scenario, list of boundaries chosen by Rust vs donor
2. Per-scenario divergence count and total compressed-size delta
3. Verify: where Rust splits differently AND donor produces smaller
   output → port the donor decision logic

Acceptance criteria

• Comparator harness in tests/ or benches/ reading decodecorpus_files/
• Report shows boundary-by-boundary diff for each scenario
• If divergences correlate with ratio loss → port the donor split logic
• If divergences are neutral (same compressed size) → close as algorithmic
  freedom

Files involved

• zstd/src/encoding/block_splitter.rs (current heuristic)
• New harness in zstd/tests/ or zstd/benches/

References

• Related to feat: block splitting for improved compression ratio #23 (feature-level block splitting)
• Donor: lib/compress/zstd_compress_superblock.c::ZSTD_splitBlock

[coderabbitai]

🔗 Related PRs

#43 - test(bench): expand benchmark parity matrix [merged]
#140 - feat(encoding): #23 add donor _fromBorders pre-split heuristic + broaden level coverage [merged]
#149 - feat(bench): #99 Rust↔FFI sequence-stream comparator [merged]
#166 - perf(fse): donor FSE_buildCTable_wksp parity — drop per-symbol Vec [merged]
#179 - perf(encoder): donor-parity greedy parse at L4 — ratio + speed win [merged]
#181 - perf(encoder): G3 — whole-block bail-out before partition split [merged]
#182 - feat(encoder): strategy-aware literal gates (G4 + G5) [merged]
#187 - perf(encoder): bump Fast window_log to 19 (donor parity, large-log-stream ratio fix) [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!