Spec: require monotonically increasing snapshot timestamp-ms in v4#16294
Spec: require monotonically increasing snapshot timestamp-ms in v4#16294stevenzwu wants to merge 1 commit into
Conversation
github-actions
Bot
added
the
Specification
stevenzwu
force-pushed
the
spec-snapshot-timestamp-monotonic
branch
2 times, most recently
from
bf46974 to
8ec0c31
Compare
| | | | _required_ | **`added-rows`** | The upper bound of the number of rows with assigned row IDs, see [Row Lineage](#row-lineage) | | ||
| | | | _optional_ | **`key-id`** | ID of the encryption key that encrypts the manifest list key metadata | | ||
|
|
||
| In format version 4 and above, a snapshot's `timestamp-ms` must be strictly greater than the `timestamp-ms` of its parent snapshot on the same branch. Root snapshots (those without a parent) are not constrained by this requirement. |
There was a problem hiding this comment.
how about tables migrated from v3 to v4 ... with broken timestamps ? do we need to say something about them ?
There was a problem hiding this comment.
Good catch. Pushed an update that explicitly carves out pre-upgrade snapshots:
In format version 4 and above, a snapshot's
timestamp-msmust be strictly greater than thetimestamp-msof its parent snapshot on the same branch. The requirement applies only to snapshots committed while the table is at format version 4 or higher; root snapshots, and snapshots that existed before a table was upgraded to v4, are not constrained.
So a table migrated from v3 with non-monotonic timestamps is conformant: the existing snapshots are grandfathered in, and only new snapshots committed at v4+ must satisfy the rule. A new v4 snapshot whose parent happens to be a pre-upgrade snapshot must still beat that parent's timestamp-ms, which will naturally fast-forward past any anomaly inherited from v3.
In format version 4 and above, a snapshot's `timestamp-ms` must be strictly greater than the `timestamp-ms` of its parent snapshot on the same branch. The requirement applies only to snapshots committed while the table is at format version 4 or higher; root snapshots, and snapshots that existed before a table was upgraded to v4, are not constrained. Adds the requirement to the Snapshots section and a new Version 4 subsection in Appendix E. No table or schema fields are added. Made-with: Cursor Model: claude-4.7-opus Co-authored-by: Cursor <cursoragent@cursor.com>
stevenzwu
force-pushed
the
spec-snapshot-timestamp-monotonic
branch
from
8ec0c31 to
7c2f7fe
Compare
2 participants
Summary
Adds a v4 spec requirement that a snapshot's
timestamp-msmust be strictly greater than the parent snapshot'stimestamp-mson the same branch. Root snapshots (with no parent) are not constrained.format/spec.mdSnapshots section: notes the v4-only requirement directly under the snapshot field table.format/spec.mdAppendix E: adds a newVersion 4subsection (aboveVersion 3) listing the snapshot-timestamp rule. This is the first v4 entry in Appendix E and is intentionally minimal\u2014other v4 changes can be appended as they land.The spec only states the requirement; it does not prescribe how writers should achieve it or enumerate downstream features that benefit from it.
No table or schema fields are added. v1\u2013v3 semantics are unchanged.
Companion PR
A reference implementation in
SnapshotProduceris proposed in #16293.Test plan
format/spec.mdrenders correctly (note added to the v1\u2013v3 Snapshots tab; newVersion 4subsection in Appendix E aboveVersion 3)