Epic: Commonware Integration — P2P Consensus & cvd Binary

[tac0turtle]

Goal

Integrate Commonware as the native consensus and P2P layer for Evolve. Deliver a new bin/cvd binary that runs a fully self-contained Evolve chain with BFT consensus, authenticated P2P networking, and validator management — no external consensus process required.

The success criteria: a user can spin up a multi-validator Evolve chain using cvd with Commonware consensus out of the box.

Reference Implementations

• Alto — Minimal blockchain built on Commonware. Shows the clean wiring pattern: Application trait impl -> simplex consensus -> marshal -> P2P channels -> archive storage. Alto has no application-layer state (no STF, no smart contracts), so it's the baseline for understanding Commonware's component model.
• Tempo — Production payment chain using Commonware consensus + Reth EVM execution. Shows how to bridge an execution engine (Reth) with Commonware consensus in a dual-runtime architecture. Relevant patterns: subblock gossip, DKG coordination, payload builder integration, consensus/execution bridge.

Current State

Evolve currently has two binaries:

• evd — Execution node exposing gRPC for an external consensus process (ev-node). No native P2P or consensus.
• testapp — Dev-mode binary with DevConsensus (mock block production, single node, no P2P).

Both use commonware-runtime for the async runtime but nothing else from Commonware.

Scope

1. Commonware Application Trait — Evolve Adapter

Bridge Evolve's STF with Commonware's consensus interface.

• Implement commonware_consensus::Application for Evolve
  • genesis() — Return genesis block from Evolve's genesis flow
  • propose() — Build a block from mempool transactions using Evolve's BlockBuilder + STF
  • Block type wrapping Evolve's Block with Commonware's Heightable, Committable, Digestible
• Implement commonware_consensus::VerifyingApplication
  • verify() — Validate proposed block (timestamp monotonicity, parent chain, tx validity)
• Implement Reporter for finalization callbacks
  • On finalization: execute block through STF, commit state changes to storage, update chain index
• Implement commonware_codec::{Encode, Decode} for Evolve's block type (or use RLP/borsh bridge)
• Define the consensus → execution boundary cleanly (what runs inside consensus vs. what runs on finalization)

2. P2P Networking

Wire Commonware's authenticated P2P for validator communication and transaction gossip.

• Configure commonware-p2p::authenticated::Network with Ed25519 validator identities
• Define P2P channel layout:
  • Consensus votes channel (high rate)
  • Consensus certificates channel (high rate)
  • Block resolver channel (medium rate)
  • Block broadcaster channel (low rate)
  • Marshal/backfill channel
  • Transaction gossip channel (not in Alto — Evolve needs mempool gossip between validators)
• Rate limiting configuration per channel
• Bootstrapper/seed node configuration
• Peer discovery and management

3. Consensus Engine Wiring

Set up Commonware's simplex BFT consensus.

• Configure commonware_consensus::simplex::Engine with Evolve's Application impl
• BLS12-381 threshold signature scheme (following Alto's vrf::Scheme pattern)
• commonware_broadcast::buffered::Engine for message buffering
• commonware_consensus::marshal::Actor for ordered finalized block delivery
• commonware_resolver for block fetching during sync
• Timeout configuration (leader timeout, notarization timeout, activity timeout)
• VRF-based leader election

4. Validator Management

• Validator key management (Ed25519 for P2P identity, BLS12-381 for consensus)
• Validator set configuration (genesis validator set)
• Key generation CLI (cvd keygen)
• Peer configuration format (YAML with validator public keys + addresses)
• Epoch transitions — Define how/when the validator set can change (fixed epoch initially, dynamic later)

5. Block Production & Execution Pipeline

Connect consensus finalization to Evolve's STF execution.

• Proposal path: Leader collects txs from mempool → builds Evolve block → proposes to consensus
• Finalization path: On finalized block → execute through STF → commit state → update index
• Mempool integration: Transaction gossip between validators, mempool fed from both P2P and JSON-RPC
• Gas/block limits: Enforce block gas limit in proposal building
• Ensure deterministic block execution (same block always produces same state root regardless of which validator executes it)

6. Storage & Persistence

• QMDB storage for application state (existing)
• Commonware archive storage for consensus data (finalized blocks, certificates)
  • Freezer tables with LZ4/zstd compression
  • Pruning configuration
• Chain state persistence (height, genesis info, validator set — extending existing ChainState)
• State sync foundation — a new validator should be able to catch up from peers

7. bin/cvd Binary

The new binary that ties everything together.

• CLI structure:
  • cvd run — Start a validator node
  • cvd init — Initialize genesis and generate config
  • cvd keygen — Generate validator keypair
  • cvd follow — Run a follower/archive node (no consensus participation)
• Configuration:
  • Validator identity (key path)
  • P2P listen address + bootstrappers
  • Consensus parameters (timeouts, epoch length)
  • Storage paths (app state, consensus archive)
  • JSON-RPC server config
  • Chain ID and genesis config
• Dual component architecture:
  • Commonware runtime for consensus + P2P
  • Evolve STF execution on finalization callbacks
  • JSON-RPC server for external queries and tx submission
• Graceful shutdown coordinating consensus, P2P, RPC, and storage

8. Follower / Archive Node

• Follower binary mode (cvd follow) that syncs finalized blocks without participating in consensus
• Block fetching from indexer or peers
• Consensus signature verification on received blocks
• Full STF re-execution for state verification
• JSON-RPC serving for read queries

9. Missing Features Gap Analysis

Features needed for a production chain that neither evd nor testapp currently provide:

• Transaction gossip protocol — Validators need to share pending txs (Alto doesn't have this since it has no tx concept; Tempo handles it via subblock gossip)
• Block gas metering in proposals — Leader must respect gas limits when building proposals
• Consensus-aware chain indexing — Index includes consensus metadata (notarizations, finalizations, validator signatures)
• Health/readiness endpoints — Operational endpoints showing consensus status, peer count, sync state
• Metrics integration — Wire Commonware's metrics trait to Evolve's observability
• Genesis ceremony tooling — Multi-validator genesis coordination (each validator contributes their key, produces shared genesis)
• DKG (Distributed Key Generation) — Required for BLS threshold signatures if using polynomial shares (Tempo has this; Alto uses pre-configured shares)

Design Considerations

• Single runtime vs. dual runtime: Alto uses a single Commonware runtime. Tempo uses two (Commonware for consensus, Tokio for Reth execution). Evolve's STF is synchronous and fast, so a single Commonware runtime (which wraps Tokio) should work. The STF executes on finalization callbacks, not in a separate runtime.
• Block execution timing: Execute blocks on finalization (not on notarization). This matches Alto's pattern and ensures only finalized state is committed.
• Namespace separation: Use a unique namespace (e.g., b"_EVOLVE") for signing domain separation, preventing signature replay across chains.
• Transaction gossip: This is the biggest gap vs. Alto. Evolve needs a dedicated P2P channel for mempool transaction propagation. The leader builds blocks from its local mempool, so all validators need to see pending transactions.
• Feature parity with evd: cvd should support everything evd does (JSON-RPC, block indexing, ETH tx format) plus native consensus. evd remains for external consensus deployments.

Success Criteria

1. cvd keygen generates validator keys (Ed25519 + BLS12-381).
2. cvd init produces genesis config for N validators.
3. cvd run starts a validator node that participates in BFT consensus with peers.
4. A 4-validator testnet produces blocks, finalizes them, and serves JSON-RPC queries.
5. cvd follow syncs a follower node from the validator set.
6. Transactions submitted via JSON-RPC to any validator are gossiped and included in blocks.
7. The chain survives 1 validator going offline (BFT tolerance: f < n/3).

References

• Commonware Monorepo — All primitives (consensus, p2p, storage, crypto, runtime)
• Alto — Minimal reference chain (simplex consensus, BLS12-381, archive storage)
• Tempo — Production chain showing execution/consensus bridge pattern
• Commonware Consensus Docs — Simplex, aggregation, marshal, Application/Automaton traits