Skip to content

Conversation

@kariy
Copy link
Member

@kariy kariy commented Sep 17, 2025

Summary

  • Introduces a declarative macro system to simplify adding new database versions
  • Reduces boilerplate from hundreds of lines to just a few lines per version
  • Maintains full backward compatibility with existing databases

Problem

Currently, adding a new database version requires manually defining entire types, even if only a few fields changed. This creates hundreds of lines of boilerplate code and makes it error-prone to add new versions.

Solution

The new versioned_type! macro automatically generates:

  • Versioned enum with all variants
  • From trait implementations for conversions
  • Compress and Decompress implementations with proper fallback chain
  • Tests for backward compatibility

Changes

  1. Added macro infrastructure (macros.rs):

    • versioned_type! macro for generating versioned enums
    • Automatic trait implementations for serialization/deserialization
    • Fallback chain for backward compatibility
  2. Migrated existing types:

    • Transaction types (V6/V7) now use the macro
    • Block header types (V6/V7) now use the macro
    • All existing functionality preserved
  3. Documentation:

    • Comprehensive README explaining the versioning system
    • Example file showing how to add new versions
    • Inline documentation for the macros

Example Usage

Adding a new version is now as simple as:

// Before: Hundreds of lines of boilerplate
// After: Just add one line
versioned_type! {
    VersionedTx {
        V6 => v6::Tx,
        V7 => v7::Tx,
        V8 => v8::Tx,  // New version added here
        V9 => katana_primitives::transaction::Tx,  // Latest
    }
}

Testing

  • ✅ All existing tests pass
  • ✅ Added tests for macro-generated code
  • ✅ Verified backward compatibility with V6 data
  • ✅ Round-trip serialization works correctly

Benefits

  1. Minimal boilerplate: Adding versions reduced from ~200 lines to ~5 lines
  2. Type safety: Compile-time verification of version compatibility
  3. Automatic migration: Conversion code is generated automatically
  4. Better maintainability: Clear separation of version-specific changes
  5. Documentation: Self-documenting version history

🤖 Generated with Claude Code

@codecov
Copy link

codecov bot commented Sep 17, 2025

Codecov Report

❌ Patch coverage is 96.57534% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.50%. Comparing base (9bde0ae) to head (4a32267).
⚠️ Report is 114 commits behind head on main.

Files with missing lines Patch % Lines
crates/storage/db/src/models/versioned/macros.rs 84.00% 4 Missing ⚠️
...rates/storage/db/src/models/versioned/block/mod.rs 98.64% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #273      +/-   ##
==========================================
+ Coverage   73.32%   75.50%   +2.18%     
==========================================
  Files         209      231      +22     
  Lines       23132    27234    +4102     
==========================================
+ Hits        16961    20564    +3603     
- Misses       6171     6670     +499     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Introduces a declarative macro system to simplify adding new database versions. The new `versioned_type!` macro automatically generates versioned enums with all necessary trait implementations, reducing boilerplate from hundreds of lines to just a few lines per version.

Key changes:
- Add `versioned_type!` macro for generating versioned enums with automatic Compress/Decompress implementations
- Migrate existing V6/V7 transaction and block types to use the new macro system
- Add comprehensive documentation and examples for adding new versions
- Include tests to verify backward compatibility and round-trip serialization

This makes database format changes much more manageable - adding a new version now only requires:
1. Incrementing CURRENT_DB_VERSION
2. Adding one line to the versioned type declaration
3. Defining only the types that changed

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@kariy kariy marked this pull request as draft September 17, 2025 21:06
@kariy kariy force-pushed the versioned-db-types branch from 4a32267 to 2576e52 Compare September 17, 2025 21:06
kariy and others added 8 commits September 17, 2025 17:14
Marks older version decompression paths as #[cold] and #[inline(never)] to optimize for the common case where most data is in the latest format. This helps the compiler generate better code for the hot path (latest version), improving decompression performance for typical database reads.

The optimization splits the decompression attempts into:
- Hot path: Latest version (most common case)
- Cold paths: Older versions (less common, marked with compiler hints)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
…ping

Extends ResourceBoundsMapping serde to support different formats based on serializer type:
- Binary formats (postcard, bincode): Uses standard enum serialization with variant tags
- Human-readable formats (JSON): Uses unified object format with all possible fields

This allows ResourceBoundsMapping to:
- Maintain backward compatibility with existing binary database format
- Support JSON deserialization from a single object containing union of all fields
- Automatically determine variant based on presence of l1_data_gas field

Added comprehensive tests for both JSON and binary serialization/deserialization.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
Introduces a procedural macro that eliminates boilerplate when creating versioned database types. The macro automatically generates version-specific structs and From implementations based on field-level attributes.

Key features:
- #[derive(Versioned)] generates version-specific modules and structs
- Field-level #[versioned()] attributes specify version-specific types
- Automatic From trait implementations using .into() for conversions
- Support for both structs and enums
- Users retain full control over conversion logic via From traits

This reduces version addition from ~200 lines to just a few attribute annotations.

Example usage:
```rust
#[derive(Versioned)]
pub struct InvokeTxV3 {
    #[versioned(v6 = "v6::ResourceBoundsMapping")]
    pub resource_bounds: ResourceBoundsMapping,
    // other fields...
}
```

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@kariy kariy force-pushed the main branch 2 times, most recently from e3bd68b to e897bbb Compare December 30, 2025 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants