[MEL] - Pt. 1 Modify the Message Extractor with Latest, Working Logic

[rauljordan]

This PR modifies the mel/ package of Nitro with all latest changes we have gotten to work in #4152, with system tests in #4152 that show we can run message extraction and validate it correctly. This PR includes the following important changes:

1. Modifications to the message extraction function so that delayed message accumulation happens later in the function. This is important for proving mode, but not as relevant for native mode
2. Modification to message hashing and RLP encoding to include all its fields, as batch posting costs are now part of MEL consensus as MEL makes this possible
3. Reworking the message extractor's finite state machine into 4 separate files that each deal with each responsibility: initialize(), processNextBlock(), reorg(), and saveMessages()
4. Make the message extractor satisfy the InboxTracker/Reader interface by implementing functions such as FindInboxBatchContainingMessage with MEL-specific logic
5. Add config flags to the message extractor with defaults

All of this is taken from #3174

[codecov]

Codecov Report

❌ Patch coverage is 0% with 306 lines in your changes missing coverage. Please review.
✅ Project coverage is 32.94%. Comparing base (26d4dc2) to head (b2afa9b).
⚠️ Report is 10 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4398      +/-   ##
==========================================
- Coverage   32.95%   32.94%   -0.02%     
==========================================
  Files         489      493       +4     
  Lines       58146    58360     +214     
==========================================
+ Hits        19164    19227      +63     
- Misses      35638    35756     +118     
- Partials     3344     3377      +33     

[github-actions]

❌ 9 Tests Failed:

Tests completed	Failed	Passed	Skipped
4233	9	4224	0

View the top 3 failed tests by shortest run time

TestManageTransactionFilterers

Stack Traces | 4.090s run time

... [CONTENT TRUNCATED: Keeping last 20 lines]
INFO [02-24|11:43:02.349] Submitted transaction                    hash=0xbd2f29c35687cae65da518298f44d48bb07fb275667ffc8a5c416531b6691396 from=0x26E554a8acF9003b83495c7f45F06edCB803d4e3 nonce=2    recipient=0x0000000000000000000000000000000000000071 value=1,000,000,000,000,000,000
INFO [02-24|11:43:02.350] Chain head was updated                   number=15  hash=19fa5f..7e0c03 root=4daac9..6d949f elapsed=1.684687ms
INFO [02-24|11:43:02.352] Imported new potential chain segment     number=802 hash=aa10fc..8c7a97 blocks=1  txs=1  mgas=0.021  elapsed=11.672ms     mgasps=1.799    triediffs=686.40KiB  triedirty=363.42KiB
INFO [02-24|11:43:02.353] Chain head was updated                   number=802 hash=aa10fc..8c7a97 root=62fea8..384edc elapsed="72.365µs"
INFO [02-24|11:43:02.355] Submitted transaction                    hash=0xfb8bcc7c790d29a2b475eb229019d8768a381e2c4509cd2d76287f4e6c3fc752 from=0x26E554a8acF9003b83495c7f45F06edCB803d4e3 nonce=497  recipient=0x0C709F340F0BB2e361229e345B7e26999d0969Ab value=1
INFO [02-24|11:43:02.361] Submitted transaction                    hash=0x54160f486fe5aa96892723f2b3e74b7040e41dfcb6fd1b69a5235a00abf9d5fb from=0x26E554a8acF9003b83495c7f45F06edCB803d4e3 nonce=498  recipient=0x0C709F340F0BB2e361229e345B7e26999d0969Ab value=1
INFO [02-24|11:43:02.365] Deploying outbox
INFO [02-24|11:43:02.369] Submitted contract creation              hash=0xf3ab169959ed3a8a63a941f2f2cf30f4a21ac55cfee441f14fed03af6f9150ce from=0x57Ff0F473737a1c161bfF9efDF016F7991585088 nonce=7    contract=0x6Fe8F13052e9d166cf09a37fb089C5c9bdc1B080 value=0
INFO [02-24|11:43:02.370] Starting work on payload                 id=0x03c9a390719776c0
INFO [02-24|11:43:02.370] Starting work on payload                 id=0x03217a050e389d93
INFO [02-24|11:43:02.371] Updated payload                          id=0x03217a050e389d93 number=8   hash=69e12b..50a620 txs=1  withdrawals=0 gas=1,491,086  fees=1.491086e-06   root=4d0e00..e90139 elapsed="594.432µs"
INFO [02-24|11:43:02.373] Updated payload                          id=0x03c9a390719776c0 number=16  hash=418ca3..3c9cb5 txs=2  withdrawals=0 gas=8,191,685  fees=8.191685e-06   root=8e5412..c32451 elapsed=2.174ms
INFO [02-24|11:43:02.373] Imported new potential chain segment     number=8   hash=69e12b..50a620 blocks=1  txs=1  mgas=1.491  elapsed=1.778ms      mgasps=838.590  triediffs=24.74KiB   triedirty=0.00B
INFO [02-24|11:43:02.373] Submitted contract creation              hash=0xfe6d8c973e90245226136e7ac1e0ba6452f25647a7c634a66fdb5c5839716aca from=0x57Ff0F473737a1c161bfF9efDF016F7991585088 nonce=19   contract=0x1174d3891010c0e64f57f3E47f1dEd82DC1DFA34 value=0
INFO [02-24|11:43:02.374] Chain head was updated                   number=8   hash=69e12b..50a620 root=4d0e00..e90139 elapsed="226.363µs"
WARN [02-24|11:43:02.377] Error getting finalizedMessageCount from syncMonitor err="block number not supported: finalized block not found"
INFO [02-24|11:43:02.373] Stopping work on payload                 id=0x03217a050e389d93 reason=delivery
�[90mTime to activate hostio-test: 506.149921ms�[0;0m
INFO [02-24|11:43:02.378] Stopping work on payload                 id=0x03c9a390719776c0 reason=delivery
--- FAIL: TestManageTransactionFilterers (4.09s)

TestVersion40

Stack Traces | 5.830s run time

... [CONTENT TRUNCATED: Keeping last 20 lines]
INFO [02-24|11:42:56.542] Deploying ospHostIo
    precompile_inclusion_test.go:94: goroutine 602158 [running]:
        runtime/debug.Stack()
        	/opt/hostedtoolcache/go/1.25.6/x64/src/runtime/debug/stack.go:26 +0x5e
        github.com/offchainlabs/nitro/util/testhelpers.RequireImpl({0x40f3bf0, 0xc01ec0ae00}, {0x40b0540, 0xc1103bd1a0}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/util/testhelpers/testhelpers.go:29 +0x55
        github.com/offchainlabs/nitro/system_tests.Require(0xc01ec0ae00, {0x40b0540, 0xc1103bd1a0}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/system_tests/common_test.go:2075 +0x5d
        github.com/offchainlabs/nitro/system_tests.testPrecompiles(0xc01ec0ae00, 0x28, {0xc110e87df8, 0x5, 0x39?})
        	/home/runner/work/nitro/nitro/system_tests/precompile_inclusion_test.go:94 +0x371
        github.com/offchainlabs/nitro/system_tests.TestVersion40(0xc01ec0ae00?)
        	/home/runner/work/nitro/nitro/system_tests/precompile_inclusion_test.go:71 +0x64b
        testing.tRunner(0xc01ec0ae00, 0x3d2adc8)
        	/opt/hostedtoolcache/go/1.25.6/x64/src/testing/testing.go:1934 +0xea
        created by testing.(*T).Run in goroutine 1
        	/opt/hostedtoolcache/go/1.25.6/x64/src/testing/testing.go:1997 +0x465
        
    precompile_inclusion_test.go:94: �[31;1m [] execution aborted (timeout = 5s) �[0;0m
INFO [02-24|11:43:01.944] Submitted transaction                    hash=0xa6010288b8c631c28f1db0732d56394b3b968488325098f3f9c157b64a8b7de2 from=0x26E554a8acF9003b83495c7f45F06edCB803d4e3 nonce=8    recipient=0x3a0a61C11D96F5B8c1492bEaA5bDAedefFff15E8 value=0
--- FAIL: TestVersion40 (5.83s)

TestVersion30

Stack Traces | 6.020s run time

... [CONTENT TRUNCATED: Keeping last 20 lines]
INFO [02-24|11:42:56.163] Stopping work on payload                 id=0x03e860d5a844f687 reason=delivery
    precompile_inclusion_test.go:94: goroutine 602157 [running]:
        runtime/debug.Stack()
        	/opt/hostedtoolcache/go/1.25.6/x64/src/runtime/debug/stack.go:26 +0x5e
        github.com/offchainlabs/nitro/util/testhelpers.RequireImpl({0x40f3bf0, 0xc01ec0ac40}, {0x40b0540, 0xc1103bcf60}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/util/testhelpers/testhelpers.go:29 +0x55
        github.com/offchainlabs/nitro/system_tests.Require(0xc01ec0ac40, {0x40b0540, 0xc1103bcf60}, {0x0, 0x0, 0x0})
        	/home/runner/work/nitro/nitro/system_tests/common_test.go:2075 +0x5d
        github.com/offchainlabs/nitro/system_tests.testPrecompiles(0xc01ec0ac40, 0x1e, {0xc158fa7db0, 0x6, 0x0?})
        	/home/runner/work/nitro/nitro/system_tests/precompile_inclusion_test.go:94 +0x371
        github.com/offchainlabs/nitro/system_tests.TestVersion30(0xc01ec0ac40?)
        	/home/runner/work/nitro/nitro/system_tests/precompile_inclusion_test.go:67 +0x798
        testing.tRunner(0xc01ec0ac40, 0x3d2adc0)
        	/opt/hostedtoolcache/go/1.25.6/x64/src/testing/testing.go:1934 +0xea
        created by testing.(*T).Run in goroutine 1
        	/opt/hostedtoolcache/go/1.25.6/x64/src/testing/testing.go:1997 +0x465
        
    precompile_inclusion_test.go:94: �[31;1m [] execution aborted (timeout = 5s) �[0;0m
WARN [02-24|11:43:01.944] Served eth_call                          reqid=11   duration=5.399825554s  err="execution aborted (timeout = 5s)"
--- FAIL: TestVersion30 (6.02s)

📣 Thoughts on this report? Let Codecov know! | Powered by Codecov