Trie hasher [POC]

[n13]

POC of what it would take to make a TrieHasher approach.

Put on top of no-length-trie branch

Wormhole tests fail

Pretty extensive code changes, not sure

• This is optimal
• This is worthwhile

So this PR is for evaluating these things.

===================================

Structured Trie Hasher

Problem

The generic hash_db::Hasher trait provides a single hash(&[u8]) -> Out method with no context about what is being hashed. When PoseidonHasher receives raw bytes, it must apply the injective byte-to-felt encoding (injective_bytes_to_felts) to safely convert arbitrary bytes into Goldilocks field elements before hashing. This encoding uses 4 bytes per felt with a terminator — it's safe for arbitrary data but expensive to prove in ZK circuits.

However, ZK-trie nodes are already felt-aligned (8-byte chunks that map directly to Goldilocks field elements). The injective encoding is redundant for this data — it doubles the number of field elements and circuit constraints for no safety benefit.

Solution

Extend the Hasher trait with context-aware hashing methods so PoseidonHasher can apply the optimal encoding for each data type.

Core API change (hash-db)

Added two methods with default implementations to the Hasher trait:

pub trait Hasher: Sync + Send {
    // ... existing methods ...
    fn hash(x: &[u8]) -> Self::Out;

    fn hash_node(encoded_node: &[u8]) -> Self::Out {
        Self::hash(encoded_node)
    }

    fn hash_value(value: &[u8]) -> Self::Out {
        Self::hash(value)
    }
}

Defaults delegate to hash, so all existing Hasher implementations (e.g. Blake2Hasher) continue to work without changes.

A TrieHasher marker trait with a blanket impl keeps call-site bounds readable:

pub trait TrieHasher: Hasher {}
impl<H: Hasher> TrieHasher for H {}

PoseidonHasher optimization (qp-poseidon)

PoseidonHasher overrides hash_node to skip injective encoding and directly convert 8-byte chunks to field elements:

fn hash_node(encoded_node: &[u8]) -> H256 {
    let felts: Vec<Goldilocks> = encoded_node
        .chunks(8)
        .map(|chunk| {
            let mut buf = [0u8; 8];
            buf[..chunk.len()].copy_from_slice(chunk);
            Goldilocks::from_u64(u64::from_le_bytes(buf))
        })
        .collect();
    H256::from_slice(&hash_to_bytes(&felts))
}

hash_value delegates to hash_for_circuit (injective encoding) since storage values are arbitrary bytes.

Inline node threshold (trie-db)

Added MAX_INLINE_NODE to TrieLayout to control the child inlining threshold:

pub trait TrieLayout {
    const MAX_INLINE_NODE: Option<u32> = None; // None = default (Hash::LENGTH)
    // ...
}

Both LayoutV0 and LayoutV1 set MAX_INLINE_NODE: Some(0) to force all children to be hashed (no inline nodes), consistent with the existing MAX_INLINE_VALUE: Some(0) policy.

HashDB routing (memory-db)

HashDB::insert now calls H::hash_value (for storage values). A new insert_node method calls H::hash_node (for encoded trie nodes). All trie-building call sites in trie-db route through the appropriate method.

Impact

• ZK circuit constraints: Roughly halved for trie node hashing — felt-aligned nodes now produce 1 felt per 8 bytes instead of 1 felt per 4 bytes with injective overhead.
• Consensus: This is a consensus-breaking change. Trie roots will differ because PoseidonHasher::hash_node produces different output than PoseidonHasher::hash. Requires genesis reset.
• Backward compatibility: All non-Poseidon hashers (e.g. Blake2Hasher) are unaffected — default implementations delegate to hash.

Changed packages

New local forks (in primitives/)

Crate	Upstream	Change
hash-db	paritytech/trie	hash_node/hash_value on Hasher, TrieHasher alias, insert_node on HashDB
memory-db	paritytech/trie	insert routes through hash_value, new insert_node routes through hash_node
trie-root	paritytech/trie	Uses hash_node for trie root, hash_value for large inline values

Modified in-repo crates

Crate	Change
trie-db	TrieLayout::Hash bound → TrieHasher, MAX_INLINE_NODE constant, ~15 call sites updated
sp-trie	Layout bounds, MAX_INLINE_NODE: Some(0), codec handles Inline(_, 0) sentinel
sp-state-machine	storage_hash/child_storage_hash → hash_value, all H: Hasher → H: TrieHasher

Companion PR (separate repo)

Crate	Change
qp-poseidon	PoseidonHasher overrides hash_node (direct felt alignment) and hash_value (injective encoding)

Test results

• All 18 qp-poseidon tests pass
• 58/62 sp-trie tests pass (4 pre-existing failures on illuzen/no-length-trie, unrelated to this PR)
• Our changes fixed 2 pre-existing test failures related to inline child nodes