fix: injective hash for trie value nodes (prevent Poseidon hash collisions)#512
Open
n13 wants to merge 3 commits intofix_genesis_accountsfrom
Open
fix: injective hash for trie value nodes (prevent Poseidon hash collisions)#512n13 wants to merge 3 commits intofix_genesis_accountsfrom
n13 wants to merge 3 commits intofix_genesis_accountsfrom
Conversation
PoseidonHasher::hash() uses bytes_to_felts_compact which is non-injective for variable-length data — distinct values can produce the same hash. With MAX_INLINE_VALUE=0, all trie leaf values are hashed externally, so this non-injectivity can cause silent data corruption. Introduce TrieLayout::hash_value() with a default that falls back to H::hash(), overridden in LayoutV0/V1 to use qp_poseidon_core::hash_bytes (injective, length-prefixed encoding). All trie-internal sites that hash values (triedbmut commit, iter_build visitors, node cache) now call L::hash_value() instead of H::hash() directly. MemoryDB wrapper gains a forced-entry fallback map so values that collide with memory-db's null_node_data are never silently dropped. StorageProof deserialization dual-inserts each node under both the standard H::hash key and the injective value key.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PoseidonHasher::hash()usesbytes_to_felts_compact, which is non-injective for variable-length data — distinct byte sequences can produce identical hashes. WithMAX_INLINE_VALUE = 0(all values hashed externally), this can silently corrupt storage by making different values look identical in the trie.TrieLayout::hash_value()— a new trait method that separates value hashing from node hashing. The default falls back toH::hash(), butLayoutV0/LayoutV1override it to useqp_poseidon_core::hash_bytes, which is injective (length-prefixed encoding). Trie node hashing remains unchanged (compact, ZK-friendly).MemoryDBagainst null-node collisions: the wrapper now includes aforcedfallback map so values matchingmemory-db's internalnull_node_datasentinel are never silently dropped onemplace.What changed and why
1.
TrieLayout::hash_value()— new trait method (trie-db/src/lib.rs)Added to the
TrieLayouttrait with a default implementation (H::hash(value)). This gives each layout the ability to override how values are hashed without affecting how trie nodes are hashed. The default preserves backward compatibility for any layout that doesn't override it.2. All value-hashing call sites updated (
trie-db)triedbmut.rs(2 sites)NodeToEncode::Nodehandling now callsL::hash_value(value)+db.emplace()instead ofdb.insert()iter_build.rs(4 sites)process_inner_hashed_valueimplementations (TrieBuilder,TrieRoot,TrieRootPrint,TrieRootUnhashed) callT::hash_value()node.rsValue::to_owned_valueusesL::hash_value()for cache consistency3.
LayoutV0/LayoutV1override (sp-trie/src/lib.rs)Both layouts override
hash_value()to callinjective_value_hash::<H>(), which usesqp_poseidon_core::hash_bytes— an injective hash function that length-prefixes the input before hashing into field elements.The
trie_root/trie_root_unhashedimplementations were also updated to usetrie_db::trie_visit(which respectsL::hash_value()) instead of the externaltrie-rootcrate functions (which hardcodeH::hash()).4.
MemoryDBwrapper hardened (sp-trie/src/lib.rs)memory_db::MemoryDB::emplace()silently drops values that exactly match itsnull_node_datasentinel (default:[0u8; 8]). WhenL::hash_value()produces a key different fromH::hash(), the inner DB'scontainscheck can fail even after a successfulemplace. The wrapper now stores such values in aforced: HashMap<H::Out, DBValue>fallback thatget/containscheck as a last resort.5.
StorageProofdual-insertion (sp-trie/src/storage_proof.rs)When deserializing a
StorageProofinto aMemoryDB, each node is now inserted under bothH::hash(node)(for standard node lookups) andinjective_value_hash(node)(for value lookups). This ensures reconstructed DBs can satisfy both access patterns.Test changes
Modified test assertions
recorder.rsrecorder_transaction_accessed_keys_worksBlake2Hasher::hash(value)assertions changed toLayout::hash_value(value)— the trie now stores the injective hash, soget_hash()returns the new hash. Addeduse trie_db::TrieLayoutimport.lib.rstest_realistic_chain_storageMemoryDBMetato theMemoryDBwrapper — this test insertsvec![0x00; 8]which collides withnull_node_data, so it needs theforcedfallback.Test-only layout (
ForceHashedValuesLayoutV1)Also overrides
hash_value()withinjective_value_hashand removes its customtrie_root/trie_root_unhashedoverrides (now inherits defaults that correctly usetrie_visit).Test results
Known remaining work