Skip to content

commitment: eliminate BranchMerger from fold→encode→write hot path#20548

Open
awskii wants to merge 3 commits intomainfrom
awskii/commitment-encode-direct
Open

commitment: eliminate BranchMerger from fold→encode→write hot path#20548
awskii wants to merge 3 commits intomainfrom
awskii/commitment-encode-direct

Conversation

@awskii
Copy link
Copy Markdown
Member

@awskii awskii commented Apr 14, 2026

Summary

Encode complete BranchData (full afterMap) instead of partial data that required a Merge step on the read path. This removes BranchMerger entirely from the hot path.

Changes

  • foldBranch(): use afterMap as the encoding bitmap instead of touchMap & afterMap — encoded branch now contains all cells
  • CollectUpdate() (immediate path): remove merger.Merge(prev, update) call; remove merger field and pool from BranchEncoder
  • encodeDeferredUpdate() (deferred path): remove merger parameter and workerMergerPool; update all callers in ApplyDeferredBranchUpdates
  • Add benchmarks: BenchmarkEncodeBranch, BenchmarkCollectUpdate_EncodeDirect

Benchmark Results (encode+merge vs encode-direct)

Cells Old (encode+merge) New (encode-direct) Improvement
2 ~174 ns/op ~118 ns/op −32%
8 ~454 ns/op ~404 ns/op −11%
16 ~868 ns/op ~813 ns/op −6%

Verification

  • All commitment tests pass (including deferred, concurrent, multi-step)
  • make lint passes
  • make erigon integration builds successfully

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes the commitment trie “fold → encode → write” path by always encoding complete BranchData (all afterMap cells), eliminating the need to merge partial branch encodings with previous data.

Changes:

  • Update foldBranch() to encode with bitmap = afterMap and to pass touchMap=afterMap so written BranchData is complete.
  • Remove BranchMerger usage from both immediate (CollectUpdate) and deferred (encodeDeferredUpdate / ApplyDeferredBranchUpdates) encoding paths, including removing the merger pool and encoder field.
  • Add benchmarks to compare encode-direct vs the prior encode+merge approach, and add a completed implementation plan doc.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
execution/commitment/hex_patricia_hashed.go Switch branch encoding bitmap to afterMap and pass complete maps into update collection.
execution/commitment/commitment.go Remove merger from encoder/deferred update hot paths and simplify worker pooling accordingly.
execution/commitment/commitment_bench_test.go Add benchmarks for EncodeBranch and end-to-end encode+merge vs encode-direct comparison.
docs/plans/completed/20260413-complete-branchdata-encoding.md Document the approach, testing strategy, and benchmark expectations/results for the change.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/plans/completed/20260413-complete-branchdata-encoding.md
awskii added 2 commits April 15, 2026 17:17
Encode complete BranchData (full afterMap) instead of partial data
that required a merge step on the read path.

Changes:
- foldBranch(): use afterMap as the encoding bitmap instead of
  touchMap & afterMap — encoded branch now contains all cells
- CollectUpdate(): remove merger.Merge(prev, update) call;
  remove merger field and pool from BranchEncoder
- encodeDeferredUpdate(): remove merger parameter and
  workerMergerPool; update all callers in ApplyDeferredBranchUpdates
- Add benchmarks: BenchmarkEncodeBranch, BenchmarkCollectUpdate_EncodeDirect

Benchmark results (encode+merge vs encode-direct):
  2 cells:  ~174 ns/op → ~118 ns/op  (-32%)
  8 cells:  ~454 ns/op → ~404 ns/op  (-11%)
  16 cells: ~868 ns/op → ~813 ns/op   (-6%)
@awskii awskii force-pushed the awskii/commitment-encode-direct branch from a8ec16d to 59eee40 Compare April 15, 2026 17:17
@awskii
Copy link
Copy Markdown
Member Author

awskii commented Apr 15, 2026

branch works multiple steps but even wit revert of f5593e6 still have gas mismatch on it after a while

@awskii
Copy link
Copy Markdown
Member Author

awskii commented Apr 15, 2026

[WARN] [04-15|19:48:00.725]   tx gas detail                          block=10666684 txIdx=165 txHash=0xbe4964f37cf3aab6 gasUsed=50636 cumGasUsed=25911890 computedCumGas=25911890 status=1
[WARN] [04-15|19:48:00.725] [4/6 Execution] Execution failed         block=10666684 txNum=944868573 header-hash=0x39c24a34a5be5f40761d27ac6f5f0e0689810afb1e6a09a6298b2ad65733200c err="invalid block, txnIdx=166, gas used by execution: 25911890, in header: 25928990, headerNum=10666684, 39c24a34a5be5f40761d27ac6f5f0e0689810afb1e6a09a6298b2ad65733200c" isForkValidation=false
[INFO] [04-15|19:48:00.739] [4/6 Execution] serial done              in=374.961457ms buf=7.4MB/512.0MB blk=10666679 blks=1 blk/s=2 txs=0 tx/s=0 gas/s=443.44M stepsInDB=0.86 step=2418.9 alloc=7.6GB sys=15.1GB isForkValidation=false isApplyingBlocks=true
[WARN] [04-15|19:48:00.740] Cannot update chain head                 hash=0x0f311e6be6e66da2a14f18916a017f0e29dc0d9ccd18321f91e76eb0ee405ab1 err="updateForkChoice: [4/6 Execution] invalid block, txnIdx=166, gas used by execution: 25911890, in header: 25928990, headerNum=10666684, 39c24a34a5be5f40761d27ac6f5f0e0689810afb1e6a09a6298b2ad65733200c"
[WARN] [04-15|19:48:00.740] error executing clstage                  app=caplin stage=ForkChoice err="failed to compute and notify services of new fork choice: failed to run forkchoice: bad block as forkchoice"

@awskii awskii marked this pull request as ready for review April 15, 2026 20:15
@AskAlexSharov
Copy link
Copy Markdown
Collaborator

blocked by:
#20593
and
#20445

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants