Documentation & Performance test

apps/increment/README.md

@@ -0,0 +1,395 @@
+# Increment
+
+A VOID Framework application demonstrating on-chain increment tracking with cryptographic proofs using sparse merkle trees. Features both in-memory and persistent database storage options.
+
+## Overview
+
+Increment is a minimal example showcasing the VOID Framework's capabilities for building verifiable off-chain state machines with flexible storage backends. The application:
+
+- Listens to `Incremented` events from an on-chain Solidity contract
+- Maintains a counter and tracks which addresses triggered each increment
+- Stores address-to-count mappings in a sparse merkle tree (in-memory or SQLite)
+- Generates and signs cryptographic witnesses of the current state
+- Provides HTTP API for querying state and generating merkle proofs
+- Supports both Publisher (full node) and Observer (light client) modes
+- Offers pluggable storage: in-memory or persistent SQLite database
+
+Users can increment the counter on-chain, then use the off-chain proofs to claim ownership of specific increment counts back on-chain.
+
+## Architecture
+
+### Storage Options
+
+**In-Memory Mode** (Default)
+- Fast, ephemeral storage
+- Suitable for development and testing
+- State is lost when the application stops
+
+**Database Mode** (SQLite)  
+- Persistent storage across restarts
+- Production-ready with ACID guarantees
+- Efficient queries with indexed lookups
+- Stores merkle tree nodes, proofs, and metadata
+
+### Components
+
+**State Management** (`state.rs`, `data.rs`, `db.rs`)
+- `State` trait: Abstract interface for both memory and database backends
+- `MemoryState`: In-memory implementation using `SparseMerkleTree`
+- `Db` + `Tx`: SQLite-backed implementation with transactional guarantees
+- Merkle tree maps count → address (20-byte leaf values)
+- `Witness`: Signed commitment to current state (count + merkle root + block height)
+- State transitions process `Incremented` events from the oracle
+
+**Modes** (`lib.rs`)
+- **Publisher Mode**: Full node that processes events, signs witnesses, and replicates state
+  - Runs oracle to fetch blockchain events
+  - Signs state commitments with private key
+  - Broadcasts proofs to observers via network channel
+  - Persists state to database (if enabled)
+- **Observer Mode**: Light client that receives signed state from publishers
+  - Connects to publisher's oracle and network endpoints
+  - Validates signed witnesses
+  - Maintains local state replica
+  - Provides same query API as publisher
+
+**HTTP Server** (`server.rs`)
+- Server-Sent Events (SSE) stream of current count
+- REST endpoints for proofs and state queries
+- All responses include block height for consistency
+- CORS enabled for browser access
+
+**Proof System** (`proof.rs`)
+- Manages current signed witness with block height
+- Serializes proofs for network replication
+- Custom binary format for efficient transmission
+- Coordinates with storage layer for persistence
+
+## Building
+
+### Compile Contracts
+
+Before running the application, compile the Solidity contracts:
+
+```bash
+nix run .#compile-contracts
+```
+
+This generates the ABI file at `contracts/build/increment_abi.json`.
+
+### Build the Application
+
+Enter the Nix development shell and build:
+
+```bash
+nix develop
+cargo build --package increment
+```
+
+## Running
+
+### Publisher Mode
+
+Run a full node that processes events and signs state:
+
+```bash
+# In-memory (ephemeral)
+cargo run --package increment -- publisher \
+  --key PRIVATE_KEY_ENV_VAR \
+  <oracle_config_path> \
+  <server_bind_address> \
+  <app_network_bind_address> \
+  <oracle_bind_address>
+
+# With persistent database
+cargo run --package increment -- publisher \
+  --key PRIVATE_KEY_ENV_VAR \
+  --db-path ./increment.db \
+  --oracle-db-path ./oracle.db \
+  <oracle_config_path> \
+  <server_bind_address> \
+  <app_network_bind_address> \
+  <oracle_bind_address>
+```
+
+Example:
+```bash
+export SIGNER_KEY="0x..."
+cargo run --package increment -- publisher \
+  --key SIGNER_KEY \
+  --db-path increment.db \
+  --oracle-db-path oracle.db \
+  oracle_config.yaml \
+  127.0.0.1:3500 \
+  127.0.0.1:5000 \
+  127.0.0.1:4000
+```
+
+### Observer Mode
+
+Run a light client that receives state from a publisher:
+
+```bash
+# In-memory
+cargo run --package increment -- observer \
+  <oracle_config_path> \
+  <server_bind_address> \
+  <app_network_endpoint>
+
+# With persistent database
+cargo run --package increment -- observer \
+  --db-path ./increment.db \
+  --oracle-db-path ./oracle.db \
+  <oracle_config_path> \
+  <server_bind_address> \
+  <app_network_endpoint>
+```
+
+Example:
+```bash
+cargo run --package increment -- observer \
+  --db-path increment_observer.db \
+  --oracle-db-path oracle_observer.db \
+  observer_config.yaml \
+  127.0.0.1:3600 \
+  http://localhost:5000
+```
+
+### Oracle Configuration
+
+The oracle config specifies which blockchain events to monitor.
+
+**Publisher Config** (full oracle config):
+```yaml
+query:
+  stream_config:
+    - ChainContractLogs:
+        rpc_url: !Ws "ws://localhost:8546"
+        api_key_env: null
+        contract_addresses:
+          - "0x5FbDB2315678afecb367f032d93F642f64180aa3"
+        event_signatures:
+          - "0x..." # Incremented event signature
+
+block:
+  max_block_size: 1000
+  block_duration: 200ms
+```
+
+**Observer Config** (blocks config):
+```yaml
+endpoint: "http://localhost:4000"
+height: 0
+public_key: "0x..." # Publisher's address for signature verification
+```
+
+## API Endpoints
+
+The HTTP server exposes the following endpoints:
+
+### `GET /`
+Health check endpoint. Returns `"OK"`.
+
+### `GET /current-count`
+Server-Sent Events (SSE) stream that emits the current count whenever it updates.
+
+**Response**: SSE stream with `Height<u64>` values (includes block height)
+
+### `GET /get-current-count`
+Get the current count as a single value with block height.
+
+**Response**:
+```json
+{
+  "block_height": 42,
+  "data": 100
+}
+```
+
+Returns `null` if no blocks have been processed yet.
+
+### `GET /get-app-proof/{height}`
+Get the signed witness (state commitment) at a specific block height.
+
+**Parameters**:
+- `height`: Block height to query
+
+**Response**:
+```json
+{
+  "signature": [/* bytes */],
+  "data": {
+    "block_height": 42,
+    "data": {
+      "count": 100,
+      "root": [/* 32 bytes */]
+    }
+  }
+}
+```
+
+Returns `null` if no proof exists at that height.
+
+### `GET /get-merkle-proof/{count}`
+Generate a merkle proof for a specific count at the latest block height.
+
+**Parameters**:
+- `count`: The increment count to prove
+
+**Response**:
+```json
+{
+  "block_height": 42,
+  "data": [
+    [/* 32 bytes - sibling hash at depth 0 */],
+    [/* 32 bytes - sibling hash at depth 1 */],
+    ...
+  ]
+}
+```
+
+The proof array length equals the tree height (64 levels). Returns `null` if no state exists yet.
+
+### `GET /get-counts/{address}`
+Get all counts associated with an address.
+
+**Parameters**:
+- `address`: Ethereum address (hex string with 0x prefix)
+
+**Response**:
+```json
+[1, 5, 42]
+```
+
+Returns an empty array if the address has no counts.
+
+## Example Workflow
+
+### 1. Increment On-Chain
+
+Call the `increment()` function on the Increment contract:
+
+```solidity
+// Emits Incremented(msg.sender)
+increment.increment();
+```
+
+### 2. Query Off-Chain State
+
+Wait for the oracle to process the event, then query the current count:
+
+```bash
+curl http://localhost:3500/get-current-count
+# Returns: {"block_height": 1, "data": 1}
+```
+
+### 3. Get Proof
+
+Fetch the signed witness and merkle proof:
+
+```bash
+# Get signed state commitment at block height 1
+curl http://localhost:3500/get-app-proof/1 > witness.json
+
+# Get merkle proof for count 1
+curl http://localhost:3500/get-merkle-proof/1 > proof.json
+```
+
+### 4. Claim On-Chain
+
+Use the witness and merkle proof to claim ownership on-chain:
+
+```solidity
+// Extract from witness.json and proof.json
+increment.claim_count(
+    count,           // From witness.data.data.count
+    signature,       // From witness.signature
+    root,            // From witness.data.data.root
+    proof            // From merkle proof.data array
+);
+```
+
+The contract verifies:
+1. The signature is from the trusted publisher
+2. The merkle proof is valid
+3. The proof shows msg.sender at the specified count
+4. Stores the claim on-chain in the `counts` mapping
+
+## Testing
+
+### Unit Tests
+
+Run the integration test suite:
+
+```bash
+cargo test --package increment
+```
+
+### E2E Tests
+
+The main test spins up a local Reth node and tests the full workflow:
+
+```bash
+# Requires Reth on PATH
+cargo test --package increment -- --ignored test_api
+```
+
+This test:
+- Deploys the Increment contract
+- Runs publisher and observer nodes with database storage
+- Performs increments
+- Verifies state synchronization
+- Tests on-chain claim verification
+
+### Performance Tests
+
+Run the performance benchmark:
+
+```bash
+cargo test --package increment -- --ignored test_performance --nocapture
+```
+
+This test:
+- Performs 100 increments (configurable)
+- Measures on-chain and off-chain processing times
+- Calculates transactions per second (TPS)
+- Displays detailed timing metrics
+- Verifies state consistency and merkle proof generation
+
+## Database Schema
+
+When using SQLite storage (`--db-path`), the following tables are created:
+
+- `increment`: Stores the current count
+- `tree_leaf`: Merkle tree leaf nodes (count → address mappings)
+- `tree_node`: Merkle tree internal nodes (computed hashes)
+- `owner`: Derived table of addresses that triggered increments
+- `count`: Join table for efficient address → counts queries
+- `current_proof`: Latest signed witness for each block height (limited buffer)
+- `latest_header`: Tracks the latest processed block height and hash
+
+All database operations are transactional, ensuring consistency even if the application crashes mid-block.
+
+## Contract Interface
+
+The Solidity contract provides:
+
+- `increment()`: Emit an increment event
+- `claim_count(count, signature, root, proof)`: Claim ownership with proof
+- `counts[count]`: Mapping of claimed counts to owners
+- `user_counts[address]`: Array of counts claimed by each user
+
+The contract uses SHA-256 for merkle tree hashing to match the Rust implementation.
+
+## Dependencies
+
+Key dependencies from `void-toolkit`:
+- `merkle`: Sparse merkle tree implementation (in-memory and database-backed)
+- `app`: Stream processing, block height tracking, and notifications
+- `network-channel`: State replication between nodes
+- `oracle`: Blockchain event sourcing with optional persistence
+
+Additional dependencies:
+- `rusqlite`: SQLite database bindings
+- `tokio`: Async runtime
+- `axum`: HTTP server framework