Migratable BCS Blob Storage POC

A proof of concept for schema-evolvable on-chain storage on Sui. The goal is to change how product data is stored and upgraded over time: keep one stable on-chain cell, treat application schemas as logical types in Move code, and migrate bytes in place when logic moves forward—without deleting objects and minting replacements.

Motivation

On Sui (like many blockchains), the shape of data stored in an object (with schema A) is fixed at publish time. Once you choose a struct layout for an object, you cannot change that object's fields, field order, or types in place. For a live product, that creates a painful upgrade path:

Define a new struct for schema B.
Migrate every on-chain instance: delete A-shaped objects, mint B-shaped objects, copy values across.
Update all interacting modules to use B instead of A.
Optionally run a big-bang migration before/during shipping new logic.

That can be expensive, risky, and does not scale well when you only need to touch a subset of instances over time.

This POC explores a different model: one stable storage proto that only holds a schema version number and a BCS bytes payload. Application code works with logical types (SchemaV1, SchemaV2, …) that are not the on-chain object shape. Upgrades rewrite the bytes inside the same cell—just-in-time on read/write, or in batch if you choose.

Goals

Goal	What we validate
Stable on-chain cell	A single `VersionedBlob { version, bytes }` type that never changes layout.
Logical schemas	`copy` / `drop` structs (no `key` / `store`) as the developer-facing API; serialize/deserialize via BCS.
Explicit migrations	Adjacent-version migration functions (e.g. v1 → v2); no reinterpretation of bytes across layouts.
JIT migration	On access, bump `version` and replace `bytes` in the same `&mut VersionedBlob`—same object id / DF slot / struct field.
Topology-agnostic core	Codec and migration do not care how you reach the blob (dynamic field, embedded field, or standalone object).
No silent corruption	Unit tests for round-trip, migration mapping, and JIT across all three attachment patterns.

Non-goals (for this POC)

Automatic schema codegen or trait-based generics in Move.
Nested Table / Bag inside blobs (primitives and vector<u8> only for now).
Batch migration tooling (JIT is enough to prove the model).
Production-ready app UI (the app/ package is a placeholder for a later visual demo).

Core model

Everything stored on chain is a VersionedBlob:

public struct VersionedBlob has store, copy, drop {
    version: u16,      // logical schema id (1, 2, 3, …)
    bytes: vector<u8>, // BCS payload for that version
}

Logical schemas (example: player_state) are ordinary structs with copy, drop only. They are wire formats, not chain objects:

Encode: std::bcs::to_bytes
Decode: sui::bcs + peel_* in exact field order
Migrate: peel source order → build target struct → to_bytes (never cast v1 bytes as v2)

Codec / JIT: The same &mut VersionedBlob is written back whether it lives in a dynamic field, a struct field, or a standalone object.

Phase	Typical API	Behavior
Running on v1	`encode_v1`, `decode_as_v1`, `write_v1`	Read/write v1 bytes; no migration
Running on v2 (current)	`encode_v2`, `decode_as_v2`, `write_v2`	`decode_as_v2` JIT-upgrades v1 blobs in place, then deserializes
Inspect legacy v1 without upgrading	`decode_as_v1(&blob)`	Immutable read; blob stays at version 1

decode_as_v1 takes &VersionedBlob (not &mut) so it never triggers migration—useful during the v1 era and for audit tooling after v2 ships.

Example: SchemaV1 → SchemaV2

SchemaV1 (v1 bytes)	SchemaV2 (v2 bytes)
`static_a: u64`	`dynamic_meta: vector<u8>` (new, default empty on migrate)
`dynamic_payload: vector<u8>`	`static_a: u64`
`static_b: u64`	`static_b: u64`
	`dynamic_payload: vector<u8>` (reordered)

Field order changes the BCS layout, so migration is a deliberate script — not a struct rename.

Full product style workflow (v1 era → v2 era)

The player_state package includes a worked example of running real logic on v1, shipping v2, and migrating storage on first v2 use (same VersionedBlob).

Module	Role
`logic_v1`	Product rules while deployed on schema v1 (`codec` v1 only)
`logic_v2`	Product rules after v2 ships (`ensure_v2` → JIT migrate, then v2 codec)
`workflow`	Thin lifecycle API tying eras together

Domain mapping (example player):

Field	v1 meaning	v2 meaning
`static_a`	score	score (preserved)
`dynamic_payload`	player name	player name (preserved)
`static_b`	level	level (preserved)
`dynamic_meta`	—	guild tag (new; empty after migrate)

Timeline:

1. Deploy package with logic_v1 + workflow
2. new_player_storage → register → v1_add_score / v1_view_*   (blob stays version 1)
3. Publish upgraded package with logic_v2
4. First v2_join_guild (or any logic_v2 call) → JIT v1→v2 in place
5. v2_add_score / v2_view on same blob                          (blob now version 2)

Run the workflow example test:

pnpm test:move:player_state   # includes workflow_tests::full_v1_era_then_v2_migration_on_same_blob

Architecture

┌─────────────────────────────────────────────────────────────┐
│  Application / example (player_state)                       │
│  SchemaV1, SchemaV2, migrate_v1_v2, codec::decode_as_v2   │
└───────────────────────────┬─────────────────────────────────┘
                            │ &mut VersionedBlob
┌───────────────────────────▼─────────────────────────────────┐
│  storage package                                            │
│  versioned_blob, topology helpers                           │
└───────────────────────────┬─────────────────────────────────┘
                            │
        ┌───────────────────┼───────────────────┐
        ▼                   ▼                   ▼
   Dynamic field      Store field         Keyed object
   on parent UID      on parent struct    { id, data }

Topology: how a blob is linked

The migratable layer only needs &mut VersionedBlob. Attachment is a separate concern:

Pattern	Description	POC module
Dynamic field	Blob stored under a key on a parent object's `UID`	`topology_dynamic_field`
Store field	Blob as a field on a `store` struct	`topology_store_field` (tests use `&mut parent.blob`)
Standalone object	`has key` wrapper with a `data: VersionedBlob` field	`topology_keyed_object`

There is no second on-chain “cell type”—only different ways to obtain a mutable reference to the same VersionedBlob.

Repository layout

migratable-storage-poc/
├── packages/
│   ├── storage/          # VersionedBlob + topology helpers
│   ├── player_state/     # Example schemas, migration v1→v2, codec
│   └── topology/         # Tests: JIT migration per attachment pattern
├── app/                  # Placeholder Vite app (future visual demo)
├── scripts/              # localnet + publish helpers
└── package.json

Adding schema v3

Add SchemaV3 with to_bytes / from_bytes.
Add migrate_v2_v3 and register 2 → 3 in player_state::migration.
Expose decode_as_v3 / write_v3 in codec.
Extend round-trip, migration, and topology JIT tests.

Prerequisites

Sui CLI (1.63+ package workflow)
pnpm
Node.js (for app/ and root devDependencies)

Workflow: test and run locally

1. Install dependencies

From the repo root:

pnpm install
cd app && pnpm install && cd ..

2. Run Move unit tests (no validator required)

Unit tests do not send transactions to any testnet, or mainnet. They use -e testnet only as the build environment so Sui can resolve framework dependencies (required when your active CLI env is localnet).

pnpm test:move              # all packages
pnpm test:move:storage      # storage only
pnpm test:move:player_state # schemas + migration + codec
pnpm test:move:topology     # field, object, DF attachment tests

3. Format Move sources

pnpm fmt

4. Local development (Docker)

Runs Postgres + Sui localnet + faucet + GraphQL indexer in Linux containers.

Requires: Docker Desktop (or Docker Engine + Compose v2).

One terminal — start stack, then setup / publish / app:

pnpm docker:up                    # detached (background)
pnpm docker:setup
pnpm publish:local:docker:fresh   # first run or after docker:up restart
pnpm dev

docker:setup waits for RPC and GraphQL, so you can run it right after docker:up.

Ports on the host:

Service	URL
JSON-RPC	http://127.0.0.1:9000
Faucet	http://127.0.0.1:9123
GraphQL / GraphiQL	http://127.0.0.1:9125/graphql

Optional: pnpm docker:logs to tail container logs; pnpm docker:up:fg if you prefer foreground compose (blocks the terminal).

docker:setup seeds a fixed dev CLI config into the container (docker/sui/seed/), points it at localnet, faucets, waits for GraphQL, writes app/.env.local.
Publish and key export use SUI_RUNTIME=docker (publish:local:docker* scripts).
The Next.js app still runs on the host; it talks to localhost:9000 / 9125.

Stop the stack: pnpm docker:down. Wipe volumes / chain state: pnpm docker:down:volumes (or pnpm docker:down -- -v).

Move unit tests still run on the host if you have any sui on PATH, or:

pnpm docker:sui -- move test -e testnet   # from packages/player_state after cd

5. Admin UI (with integrated keypair logic for txn submission)

pnpm use:localnet    # also writes app/.env.local with your active CLI key
pnpm faucet
pnpm publish:local   # refreshes deployed.json + .env.local
# If the app reports ArityMismatch on create, the chain has an older package:
pnpm publish:local:fresh
pnpm dev             # http://127.0.0.1:5174

The app signs transactions with the same address you fauceted and published with (sui client active-address). scripts/write-dev-signer.sh exports that key into app/.env.local (gitignored). Restart pnpm dev after regenerating .env.local.

No browser wallet is required. Do not commit .env.local or use this pattern outside local dev.

6. GraphQL + indexer exploration (optional, but used in the dapp above)

Run a local indexer and GraphQL API to see how chain data is exposed for queries—and how that differs from reading logical player fields in this POC.

With Docker (recommended): pnpm docker:up already includes indexer + GraphQL. After pnpm docker:setup and creating players in the app:

pnpm graphql:wait                    # wait until indexer is ready
pnpm graphql:query graphql/queries/02-owned-player-storage.graphql
pnpm graphql:explore                 # GraphQL json vs BCS-decoded score/name/level

GraphiQL: http://127.0.0.1:9125/graphql
Example queries: graphql/queries/*.graphql
More query notes: graphql/README.md

Indexer and data lookup: standard object vs VersionedBlob

The indexer indexes Move object types and owners, not your application’s logical schemas inside bytes. That split is the main thing to validate with GraphQL in this POC.

Standard pattern — product fields are the on-chain struct (e.g. struct Player { score, name, level }):

GraphQL can filter: objects(filter: { type: "0x..::module::Player", owner: "0x.." })
contents.json may expose fields directly:

{
  "score": "100",
  "name": "alice",
  "level": "5"
}

You can read score/name/level from GraphQL without custom decoding (subject to what the RPC/indexer serializes for that type).

This POC — wrapper object + opaque blob (PlayerStorage holds VersionedBlob):

On-chain layout:

PlayerStorage {
  id: UID,
  data: VersionedBlob { version: u16, bytes: vector<u8> }  // BCS payload for schema v1/v2
}

GraphQL contents.json for PlayerStorage looks more like:

{
  "id": { "id": "0x..." },
  "data": {
    "version": 1,
    "bytes": [100, 0, 0, 0, 0, 0, 0, 0, 11, 97, 108, 105, 99, 101, ...]
  }
}

Question	GraphQL / indexer alone	Extra step
Find all `PlayerStorage` for my address?	Yes — `filter.type` on Move type	—
Object id + blob wire version (`data.version`)?	Yes — in `contents.json`	—
Logical score / name / level (schema v1)?	No — inside `bytes`	BCS decode (`pnpm graphql:explore`, app `parseStorage`, or `contents.bcs` / `moveObjectBcs`)
Filter players by score above 100?	No — not indexed as columns	Custom indexer, off-chain DB, or denormalized fields
Query standalone `VersionedBlob` objects?	Only if you own them as top-level objects	In this POC the blob is embedded in `PlayerStorage.data` (topology: store field)

The indexer knows Move types (PlayerStorage, VersionedBlob) but not logical schema versions (v1 field order inside bytes). That mapping lives in player_state::schema_v1 / codec—the same path the admin app uses after getObject or GraphQL fetch.

JSON-RPC vs GraphQL: The full node (http://127.0.0.1:9000) returns the same nested struct shape; GraphQL adds type/owner filters and transaction history once the indexer catches up (a few seconds after each tx). Neither API exposes logical player columns without decoding bytes.

Practical takeaway: Migratable storage keeps one stable cell on chain; discovery (by type, owner, id) looks like a normal object lookup, but semantic fields stay an application concern unless you add a higher layer (custom indexer tables, denormalized Move fields, or always decode BCS client-side).

Scripts reference

Script	Command	Purpose
`test:move`	`pnpm test:move`	Run all Move unit tests
`test:move:storage`	per-package	Storage package tests
`test:move:player_state`	per-package	Schema / migration / JIT tests
`test:move:topology`	per-package	Attachment-pattern tests
`docker:up`	`pnpm docker:up`	Start stack detached (background)
`docker:up:fg`	`pnpm docker:up:fg`	Start stack in foreground (blocks terminal)
`docker:logs`	`pnpm docker:logs`	Tail compose logs
`docker:down`	`pnpm docker:down`	Stop Docker stack
`docker:down:volumes`	`pnpm docker:down:volumes`	Stop stack and remove volumes
`docker:setup`	`pnpm docker:setup`	Configure CLI, faucet, GraphQL wait, `.env.local`
`docker:sui`	`pnpm docker:sui -- …`	Run `sui` CLI in the container
`publish:local:docker`	`pnpm publish:local:docker`	Publish via Dockerized CLI
`localnet`	`pnpm localnet`	Native ephemeral localnet (host `sui`)
`localnet:graphql`	`pnpm localnet:graphql`	Native localnet + indexer + GraphQL (host Postgres)
`graphql:query`	`pnpm graphql:query <file> [objectId]`	Run a query from `graphql/queries/`
`graphql:wait`	`pnpm graphql:wait [txDigest]`	Wait for GraphQL/indexer
`graphql:explore`	`pnpm graphql:explore [objectId]`	Compare GraphQL json vs decoded blob
`localnet:isolated`	`pnpm localnet:isolated`	Validator config in `.sui-local/` (needs working `sui genesis`)
`localnet:fresh`	`pnpm localnet:fresh`	Re-init isolated `.sui-local/` genesis
`use:localnet`	`pnpm use:localnet`	Switch CLI to `http://127.0.0.1:9000`
`faucet`	`pnpm faucet`	Fund active address from local faucet
`publish:local`	`pnpm publish:local`	Ephemeral publish to localnet
`dev:signer`	`pnpm dev:signer`	Export active CLI key → `app/.env.local`
`fmt`	`pnpm fmt`	Prettier + Move plugin
`dev`	`pnpm dev`	Next.js admin UI for `app/` (port 5174)

Risks and limitations

BCS field order is the wire format. Reordering Move struct fields without bumping version corrupts existing blobs.
Migrations are hand-written. Tests and golden byte vectors are the safety net; mistakes fail silently at runtime.
Package upgrades still require publish. This POC removes delete-and-mint of storage objects, not the need to ship new Move code.
Linear version chain in POC. Only adjacent steps (n → n+1) are registered; skipping versions would need an explicit path table.

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
app		app
docker/sui		docker/sui
graphql		graphql
packages		packages
scripts		scripts
.dockerignore		.dockerignore
.gitignore		.gitignore
README.md		README.md
bash		bash
docker-compose.yml		docker-compose.yml
migratable-storage-poc@		migratable-storage-poc@
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
pnpm-workspace.yaml		pnpm-workspace.yaml
sui		sui

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Migratable BCS Blob Storage POC

Motivation

Goals

Non-goals (for this POC)

Core model

Example: SchemaV1 → SchemaV2

Full product style workflow (v1 era → v2 era)

Architecture

Topology: how a blob is linked

Repository layout

Adding schema v3

Prerequisites

Workflow: test and run locally

1. Install dependencies

2. Run Move unit tests (no validator required)

3. Format Move sources

4. Local development (Docker)

5. Admin UI (with integrated keypair logic for txn submission)

6. GraphQL + indexer exploration (optional, but used in the dapp above)

Indexer and data lookup: standard object vs VersionedBlob

Scripts reference

Risks and limitations

About

Uh oh!

Releases

Packages

Uh oh!

Contributors

Uh oh!

Languages

Folders and files

Latest commit

History

Repository files navigation

Migratable BCS Blob Storage POC

Motivation

Goals

Non-goals (for this POC)

Core model

Example: SchemaV1 → SchemaV2

Full product style workflow (v1 era → v2 era)

Architecture

Topology: how a blob is linked

Repository layout

Adding schema v3

Prerequisites

Workflow: test and run locally

1. Install dependencies

2. Run Move unit tests (no validator required)

3. Format Move sources

4. Local development (Docker)

5. Admin UI (with integrated keypair logic for txn submission)

6. GraphQL + indexer exploration (optional, but used in the dapp above)

Indexer and data lookup: standard object vs VersionedBlob

Scripts reference

Risks and limitations

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors

Uh oh!

Languages

Packages