Skip to content

docs: Skippable Frame Magic Allocations registry + bilateral hygiene policy #176

@polaz

Description

@polaz

⚠️ Deferred — post-Phase 6 drop-in parity (2026-05-19)

Project priority sequence per #28: complete encoder rewrite (#111 incl. #23) → speed/ratio optimizations (#178, #180) → params API (#27) → magicless (#26) → Phase 6 C-ABI / CLI drop-in (#126/#127/#128/#130/#131/#132) → THEN this track. lsm-tree bilateral coordination accepted 2026-05-18 — commitment preserved, but execution defers until drop-in parity ships. Pre-Phase-6 work on this issue will not be scheduled.


⚠️ Feature gate context: the registry README section added by this issue is unconditional documentation. What it documents (the typed SkippableFrame API and adjacent storage-format extensions) is gated behind the lsm Cargo feature (default off) per #171/#172/#173/#174/#175/#177. The README MUST state this explicitly so readers know they need cargo add structured-zstd --features lsm to use the registered magic-variant allocations.


Context

Per DEC-3 + DEC-4 (bilateral agreement with lsm-tree, 2026-05-18): magic variants in the RFC 8878 skippable-frame range 0x184D2A50..5F are an application-protocol-level concern, not a structured-zstd-level concern. The crate never bakes specific variant values into source. But the registry of who allocated what variant for which downstream format needs to live somewhere — otherwise two consumers might independently pick the same variant for incompatible payloads.

This issue is the registry: a README section in this repo that enumerates known allocations and documents the bilateral hygiene policy. Ships in PR-A alongside #171 (typed SkippableFrame).

Content (proposed README section)

## Skippable Frame Magic Allocations

RFC 8878 reserves 16 magic numbers (`0x184D2A50..0x184D2A5F`) for
"skippable frames" — opaque user-defined payloads that compliant
zstd decoders skip past. The `structured-zstd` crate provides a
typed [`SkippableFrame`](crate::skippable::SkippableFrame) API for
emitting and parsing these but **does not allocate any variant**.

Allocations are an application-protocol concern. Downstream consumers
embedding metadata in zstd streams should register their variants
here to prevent collisions.

### Allocated variants

| Variant | Magic       | Consumer                      | Purpose                                        | Spec |
|---------|-------------|-------------------------------|------------------------------------------------|------|
| 0       | 0x184D2A50  | lsm-tree wire format v1       | MetadataFrame (AEAD header for encrypted blocks)| <lsm-tree LSM-T1> |
| 1       | 0x184D2A51  | lsm-tree wire format v1       | BodyFrame (encrypted payload)                  | <lsm-tree LSM-T1> |
| 2       | 0x184D2A52  | lsm-tree wire format v1       | EccFrame (reserved for future ECC layer)       | <lsm-tree LSM-T5> |
| 3–15    | 0x184D2A53..5F | reserved by lsm-tree v1     | future versions / extensions                   ||

### Allocation policy

- A consumer wishing to use a previously-unallocated variant SHOULD file a PR
  against this table referencing the downstream wire-format spec.
- Variants allocated to a consumer are reserved as a contiguous range owned by
  that consumer (e.g. lsm-tree owns 0x50..5F in v1); the consumer may
  sub-allocate inside that range without coordinating with structured-zstd.
- If a consumer abandons or supersedes a variant allocation, file a PR to
  release it back to the unallocated pool.
- `structured-zstd` is the registry steward but does not validate or enforce
  semantics — that's the consumer's responsibility.

What this is NOT

  • Not code coupling. structured-zstd's SkippableFrame::new accepts any magic_variant: u8 in 0..=15 and validates only that bound. The README registry is informational.
  • Not a CHANGELOG entry per allocation — changes are documentation-only and don't trigger releases.
  • Not a wire-format spec. Each registered allocation links out to its consumer's spec (lsm-tree's wire-format issue, anyone else's equivalent).

Acceptance criteria

  • New section "Skippable Frame Magic Allocations" added to README.md (or docs/skippable.md if README hygiene prefers).
  • Initial table populated with lsm-tree's v1 reservation (variants 0/1/2 + reserved 3-15).
  • Allocation policy documented.
  • Cross-link from SkippableFrame rustdoc → registry section.

Estimated size

~30 lines of markdown. Lands with PR-A (#171 + this issue).

Related


ADDENDUM (2026-05-18): feature gating context

The registry README section is unconditional documentation — it lives in the repo regardless of which features are built. But what it documents (the typed SkippableFrame API in zstd::skippable) is gated behind the lsm Cargo feature (default off) per #171.

The README section should explicitly note:

  • "These APIs require enabling the lsm Cargo feature in your Cargo.toml."
  • "The C FFI cdylib build remains strict drop-in for donor libzstd v1.5.7 regardless of which Rust features are enabled — magic-variant allocations affect only Rust-side typed wrappers, not the cdylib symbol surface."

This keeps the bilateral hygiene policy and consumer-discoverability of the registry while making it unambiguous that no part of this work breaks the drop-in C FFI contract.


Bilateral cross-reference

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3-lowLow priority — nice to haveenhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions