Summary
Chaingraph can save a block without saving or linking every transaction in that block when a transaction was previously announced but not yet written to the database.
Impact
Affected blocks can appear in block, but some of their transactions are absent from transaction entirely, and therefore absent from block_transaction. Queries over block.transactions and block.encoded_hex then silently return incomplete block data.
Observed example
On chaingraph.pc3.paryonusd.com, block height 943580 is present but transaction 094f0c3a0f6990f99486603ea3c1b47c0287823589cda6cbd36ccfda5fa47535 was missing from pc3 while present on gql.chaingraph.pat.mn.
A block-size audit over pc3 found 24 affected blocks and 130 missing block transactions in the scanned data:
- 128 missing from the
transaction table entirely
- 2 present in
transaction/node_transaction, but missing from block_transaction
Suspected cause
The agent transaction cache tracked transactions that had merely been seen/announced as well as transactions written to the DB. saveBlock classified block transactions using cache membership rather than whether the cache entry had db === true.
That means this flow could commit a partial block:
- Transaction is announced by
inv, creating a cache entry with db: false.
- A block containing that transaction is received before the full transaction is saved.
saveBlock treats the cached transaction as already saved.
- The transaction is omitted from the transaction insert batch.
- The SQL join from block transaction hashes to
transaction finds no row.
- The block and whichever transactions did join are committed.
Expected behavior
Block ingestion should be atomic at the block level: either all block transactions are saved and all block_transaction rows are linked, or the transaction rolls back and the agent retries/recovers.
Proposed fix
- Treat only cache entries with
db === true as already saved.
- Insert all other block transactions before linking the block.
- Add a post-insert invariant that verifies the number of joined and linked block transactions equals the block transaction count before commit.
- Add an e2e regression where a transaction is announced, requested, not provided, and then received only via block.
Summary
Chaingraph can save a block without saving or linking every transaction in that block when a transaction was previously announced but not yet written to the database.
Impact
Affected blocks can appear in
block, but some of their transactions are absent fromtransactionentirely, and therefore absent fromblock_transaction. Queries overblock.transactionsandblock.encoded_hexthen silently return incomplete block data.Observed example
On
chaingraph.pc3.paryonusd.com, block height943580is present but transaction094f0c3a0f6990f99486603ea3c1b47c0287823589cda6cbd36ccfda5fa47535was missing from pc3 while present ongql.chaingraph.pat.mn.A block-size audit over pc3 found 24 affected blocks and 130 missing block transactions in the scanned data:
transactiontable entirelytransaction/node_transaction, but missing fromblock_transactionSuspected cause
The agent transaction cache tracked transactions that had merely been seen/announced as well as transactions written to the DB.
saveBlockclassified block transactions using cache membership rather than whether the cache entry haddb === true.That means this flow could commit a partial block:
inv, creating a cache entry withdb: false.saveBlocktreats the cached transaction as already saved.transactionfinds no row.Expected behavior
Block ingestion should be atomic at the block level: either all block transactions are saved and all
block_transactionrows are linked, or the transaction rolls back and the agent retries/recovers.Proposed fix
db === trueas already saved.