Skip to content

feat(flashblocks): add streaming state root calculation infrastructure#635

Closed
Farukest wants to merge 3 commits intobase:mainfrom
Farukest:feature/streaming-state-root-622
Closed

feat(flashblocks): add streaming state root calculation infrastructure#635
Farukest wants to merge 3 commits intobase:mainfrom
Farukest:feature/streaming-state-root-622

Conversation

@Farukest
Copy link
Copy Markdown
Contributor

@Farukest Farukest commented Jan 31, 2026

Summary

  • Adds StateRootTask module for streaming state root calculation during transaction execution
  • Introduces --flashblocks.streaming-state-root CLI flag to enable the feature
  • Implements channel-based architecture to stream state updates to a background task
  • Reduces finalization latency by pre-fetching trie nodes incrementally

Test plan

  • Verify build compiles without warnings
  • Test with --flashblocks.streaming-state-root=true flag
  • Benchmark finalization latency with streaming vs synchronous state root

Closes #622

@Farukest
feat(flashblocks): add streaming state root calculation
94e6146 
Implement a background task that incrementally computes state root
during transaction execution to reduce finalization latency.

- Add StateRootTask module for streaming state updates
- Add --flashblocks.streaming-state-root CLI flag
- Add finalize_payload_with_streaming and build_block_with_state_root
- Pre-fetch trie nodes and build sparse trie incrementally

Closes base#622
@cb-heimdall
Copy link
Copy Markdown
Collaborator

cb-heimdall commented Jan 31, 2026 •
edited
Loading

🟡 Heimdall Review Status

Requirement Status More Info
Reviews 🟡 0/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 0
Sum 1

@Farukest Farukest mentioned this pull request Jan 31, 2026
Closed
@mw2000 mw2000 self-requested a review February 2, 2026 17:17
@mw2000
Copy link
Copy Markdown
Contributor

mw2000 commented Feb 3, 2026

@Farukest took a first pass at the code, for a complete implementation we'd need to have trie node prefetching implemented.

@Farukest
feat(flashblocks): implement trie node prefetching for streaming stat…
f060f0c 
…e root

This implements trie node prefetching as part of the streaming state root
calculation feature. When state updates arrive during transaction execution,
background tasks are spawned to pre-warm the trie cache by computing storage
roots for touched accounts.

Key changes:
- Add spawn_prefetch_tasks() to handle async prefetch spawning
- Add prefetch_storage_root() for cache warming via storage root computation
- Add max_prefetch_tasks config option (default: 64 concurrent tasks)
- Use Arc-wrapped provider for sharing with prefetch tasks
- Add atomic counter to track and limit active prefetch tasks
- Add tests for prefetch enabled/disabled scenarios

This reduces finalization latency by ensuring trie nodes are already in
memory when the final state root computation happens, rather than waiting
for disk I/O at that critical moment.
@Farukest
Copy link
Copy Markdown
Contributor Author

Farukest commented Feb 3, 2026 •
edited
Loading

@Farukest took a first pass at the code, for a complete implementation we'd need to have trie node prefetching implemented.

@mw2000 handled 👍
thank you

@Farukest
Merge origin/main into feature/streaming-state-root-622
76380e6 
Resolve merge conflicts after op-rbuilder to builder-core refactoring:
- Resolve config.rs content conflict
- Move state_root_task.rs to new crates/builder/core/src/flashblocks/ location
- Update flashblocks/mod.rs to include state_root_task module
- Update lib.rs to export StateRootTask types
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 5, 2026

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the Stale label Mar 5, 2026
danyalprout pushed a commit that referenced this pull request Mar 5, 2026
@tynes
specs: fix superchain token bridge naming (#635)
08c02e1 
* specs: rename SuperchainERC20Bridge to SuperchainTokenBridge

* toc: fix
mw2000 pushed a commit that referenced this pull request Mar 7, 2026
@fakedev9999
refactor: refactor run_range_program (#635)
40e3498 
* refactor: refactor run_range_program

* fix

* fmt

* chore: bump celestia range elf

* chore: bump elfs
mw2000 pushed a commit that referenced this pull request Mar 9, 2026
@fakedev9999 @mw2000
refactor: refactor run_range_program (#635)
b5c902b 
* refactor: refactor run_range_program

* fix

* fmt

* chore: bump celestia range elf

* chore: bump elfs
@refcell refcell removed the Stale label Mar 9, 2026
mw2000 pushed a commit that referenced this pull request Mar 16, 2026
@fakedev9999 @mw2000
refactor: refactor run_range_program (#635)
61637cc 
* refactor: refactor run_range_program

* fix

* fmt

* chore: bump celestia range elf

* chore: bump elfs
haardikk21 pushed a commit that referenced this pull request Mar 17, 2026
@fakedev9999 @mw2000
refactor: refactor run_range_program (#635)
2851149 
* refactor: refactor run_range_program

* fix

* fmt

* chore: bump celestia range elf

* chore: bump elfs
mw2000 pushed a commit that referenced this pull request Mar 19, 2026
@fakedev9999 @mw2000
refactor: refactor run_range_program (#635)
0fed4a5 
* refactor: refactor run_range_program

* fix

* fmt

* chore: bump celestia range elf

* chore: bump elfs
mw2000 pushed a commit that referenced this pull request Mar 23, 2026
@fakedev9999 @mw2000
refactor: refactor run_range_program (#635)
9fe31cd 
* refactor: refactor run_range_program

* fix

* fmt

* chore: bump celestia range elf

* chore: bump elfs
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 24, 2026

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions Bot added the stale Flag: Stale issue or PR label Mar 24, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 25, 2026

This pull request was closed because it has been inactive for 5 days since being marked as stale.

@github-actions github-actions Bot closed this Mar 25, 2026
mw2000 pushed a commit that referenced this pull request Apr 22, 2026
@fakedev9999
refactor: refactor run_range_program (#635)
578b744 
* refactor: refactor run_range_program

* fix

* fmt

* chore: bump celestia range elf

* chore: bump elfs
mw2000 added a commit that referenced this pull request Apr 24, 2026
@mw2000 @fakedev9999
@yuwen01 @0xernesto @ctian1 @leruaa @bxue-l2 @seolaoh @theochap @vcastellm @fyInALT @Farhad-Shabani @kien-rise @TiesD @claude @github-actions @jackchuma @0x00101010 @sisyphus-dev-ai @refcell @niran @danyalprout
[backport] feat: Imports op-succinct as base-succinct (#2333)
162f87c 
* feat: improve fault proof (#563)

* feat: improve game resolution

* feat(fault-proof): separate bond claiming by mode

* docs: fix typo

* feat: set jemalloc as global allocator

* feat: update to support celestia indexer changes (#561)

* feat: update to support celestia indexer changes

See https://github.com/celestiaorg/optimism/pull/502.
Now handles batcher that switch between DA layers.

* chore: fix logger init

* chore: bump hana for v4 compatibility

* chore: bump celestia range elf

* chore: pin nightly rust toolchain (#580)

* dummy

* pin nightly version

* feat: audit fix (#575)

* tests pass, fixes in

* proposer is running locally

* fix test, still broken though

* more detailed error

* jank proxy

* wrapper solution instead

* cleanup

* fix silly errors;

* tweak forge test

* docs for dgfProposeL2Output;

* spot=false

* bump elf

* nits + docs

* add audit + pin nightly toolchain

* chore: unify binary name for celestia docker image (#577)

* feat: improve op-succinct-lite (#583)

* chore: add features support to deploy-fdg-contracts in justfile

* feat(contracts): add missing getters for public values

* fix: use end exclusive range (#585)

* fix: end exclusive range

See https://docs.celestia.org/how-to-guides/blobstream-proof-queries.

* refactor(scripts): host method for starting block

* refac: handle when None returned

* change for fetch-l2oo-config as well

* chore: bump hana (#578)

* feat: add cycles + prover gas metrics (#579)

* feat: add cycles + prover gas metrics

Also improved logging for better vis and debugging.

* chore: make fmt happy

* chore: apply comments

* docs: update spn usage (#592)

* docs: make update parameters flow more clear (#590)

* fix

* doc nits

* fix: kzg verification 3.1.0 (#594)

* fix

* refresh gh

* chore: range proof insertion log to debug

* docs: L2 node latency + cleanup (#598)

* add dev section to readme

* doc pass

* chore: bump to op-contracts v3.0.0 (#600)

* test: fix e2e test (#605)

* fix(validity): request deadline (#604)

* fix(validity): request deadline

* fix(fault-proof): request deadline

* fix: add 1T gas limit

The DEFAULT_GAS_LIMIT is 1B.

* fix: add 4h timeout to agg proof request as well

---------

Co-authored-by: fakedev9999 <taehoon@succinct.xyz>

* feat: add op-celestia-indexer to docker compose (#601)

* docs: add missing CELESTIA_INDEXER_RPC

* feat: add op-celestia-indexer to docker compose

* docs: update docs

* fix healthcheck

* docs: add warning for proof window

* docs: udpate start l1 block and op-celestia-indexer

* fix(contracts): upgrade, missing proxy in calldata for proxy admin managed instance (not naked proxies) (#606)

* fix(contracts): upgrade, missing proxy in calldata for proxy admin managed instance (not naked proxies)

* chore: forge fmt

* refac(contracts): use abi.encodeCall

---------

Co-authored-by: emiliano-conduitxyz <emiliano@conduit.xyz>

* feat: support eigenda (#572)

* feat: support eigenda

Bumps sp1, hana, hokulea for kona-client/v1.0.2 with MSRV 1.86.

* ci: install sp1 toolchain

* chore: bump elfs

* fix: write proof to stdin for proof agg

* fix: bump hokulea + hardcoded vkey + defer verification

* fmt

* bump elfs

* feat: docker for eigenda

* refac: extract proof bytes from witness data

* refac: remove executor placeholder

* chore: download srs from proxy image

* docs: add eigenda docs

* chore: bump hokulea

* make ci green

* skip canoe proof generation

* chore: bump hokulea

* bump hokulea to support mock mode for sp1-cc proving

* fix: reduce RPC concurrency to prevent rate limit errors (#608)

* chore: remove unused code

* fix: reduce RPC concurrency to prevent rate limit errors

* chore: bump hana to v1.0.0 (#612)

* chore: add missing eigenda flag (#610)

* docs: make altda deployments more clear (#615)

* feat: spawn defense tasks concurrently (#614)

* feat: spawn defense tasks concurrently

* fix: typo

* feat: limit concurrent proving tasks count

* feat: add `count_active_defense_tasks()`

* fix(fault-proof): filter by configured game type (#616)

* fix(fault-proof): filter by configured game type

* test: add e2e test for game type transition

* fix bindings

* forge fmt

* feat: add env-configurable proof fulfillment strategies (#618)

* feat: add env-configurable proof fulfillment strategies

* make ci happy

* make ci happy2

* chore: bump to v3.2.0 (#619)

* ci: enable release tagging for lite docker images (#620)

* chore: bump hana (#621)

* chore: bump hana

* chore: bump to v3.2.1.

* chore: bump hana to v1.0.1

* feat: get execution stats from network when not in mock mode (#622)

* ci: add cycle count diff ci for celestia range elf (#573)

* ci: add cycle count diff ci for celestia range elf

* chore: increase block selection offset to 2 hours

This is a simple solution for Celestia Blobstream availability

* ci: add elf label

* fix to use finalized l2 block

* fix typo

* chore: bump hokulea (#609)

* chore: bump hokulea

* ci: add eigenda to elf ci

* ci: add cycle count diff check for eigenda

* ci: fix srs error

* fix copy srs

* fix

* OP_SUCCINCT_MOCK=true for sp1-cc

* fix

* chore: bump eigenda elf (#623)

* bump elfs

* chore: clamp highest provable L2 block

This avoids proving uncommitted Celestia ranges.

* docs: fix link to not use absolute path (#625)

This prevents 404s when the book is hosted under a sub-path.

* fix: bump hokulea version and fix only ethda problem (#626)

* bump hokulea version and fix only ethda problem

* name nit

* rename eigenda_blob_provider to eigenda_preimage_provider

* update elf

* restore todo

* chore: bump elf

---------

Co-authored-by: BowenX <bxue@eigenlabs.org>

* chore: bump hana (#630)

* chore: bump hana to v1.1.0-mocha

* chore: bump to v3.2.2

* chore: bump elfs

* feat: skip CHALLENGER_WINS game resolution in proposer mode (#632)

Added a condition to skip resolution when the parent game status is CHALLENGER_WINS, since such games will be ultimately resolved as CHALLENGER_WINS. Proposers have no incentive to resolve these games.

Co-authored-by: seolaoh <osa8361@gmail.com>

* fix(fault-proof): fix proposer with canonical chain tracking (#613)

* fix(fault-proof): fix proposer with canonical chain tracking

* refactor: improve game resolution

* refactor: improve game credit claim

* refactor: improve game defense

* chore: add comments

* chore: rename to update_canonical_head_if_further

* chore: rename to is_XXX for boolean returning func

* refactor: remove early return from `refresh_state`

* fix

* fix

* refactor: refactor fix-proposer PR (#627)

* move run() to the top

* move state init from refresh state to proposer init

* huge refac

* remove is_finalized

* remove update_canonical_head_if_further

* move below new

* switch orders

* compute canonical head

* remove state snapshot

* typo

* refac remove subtree

* fmt

* update fetch_game

* rename to sync_state

* remove is_descendant_of_anchor

* refac remove_subtree

* more refac (#628)

* wip

* remove for claim_bond_for_game

* fix

* clippy

* fix

* fix

* fix

* clippy

* fix

* refac

* fix: drop if parent game is not in cache

* switch orders

* fix is_parent_resolved

* update testing doc

* remove cursor and set_cursor

* improve fetch_game

* improve sync_games

* misspell

* test: add comments to e2e tests

* feat: optionally request proofs on spn mainnet (#587)

* bump sp1 sdk

* add support for KMS requester

* update binaries

* use local signer

* use auction strategy

* fix tests and binaries

* fmt

* make timeout 1 hour

* fix cycle/gas limits

* add env vars for more prove params

* whitelist env var in validity proposer

* whitelist for faul-proof

* bump sp1 sdk & elfs

* don't split if cancelled

* sqlx + add auction_timeout env var

* fix proof request cancellation

* fix sqlx

* return early

* fix conditional check

* fix cancel logic

* return early if cancelled

* fix log

* fix sqlx

* add min auction period to requests

* fix: build errors

* feat: determine network mode from fulfillment strategies

* fix: typos

* fix: strip "0x"

* fix: set max price per pgu to 300,000,000

* fix: handle UnspecifiedFulfillmentStrategy

* fix: add handle_cancelled_request()

* fix: configure min auction period with a env var

* fix: cancel requests only if NetworkMode is Mainnet

* chore: add the new env vars in the docs

* feat: refactor network signer creation to a util fn

* fix: feedbacks

* chore: bump SP1 to v5.2.2

* fix: pin SP1 version

* chore: update ELFs

* fix: rebase

* fix: Docker tag

* chore: update ELFs again

* fix: typo

* fix: rebase

---------

Co-authored-by: leruaa <aurelien.catinon@gmail.com>

* fix: retain canonical head when pruning resolved games (#634)

* fix: retain canonical head when pruning resolved games

* clippy

* feat: resume fast finality proving after restart (#637)

* feat: range proof gas splitting (#576)

* tests pass, fixes in

* proposer is running locally

* fix test, still broken though

* more detailed error

* jank proxy

* wrapper solution instead

* cleanup

* fix silly errors;

* tweak forge test

* docs for dgfProposeL2Output;

* works locally

* cleanup

* forge fmt

* chore: update ELFs

* feat: add a range limit when computing ranges by gas

* fix: fmt

* feat: rename gas_limit to evm_gas_limit

* fix: revert RANGE_PROOF_INTERVAL default value

* fix: rename RANGE_PROOF_GAS_LIMIT to RANGE_PROOF_EVM_GAS_LIMIT

* feat: replace tuples by Range type

* chore: update comment

* chore: add `single_small_block` test

* fix: comment

* fix: add a warning if no ranges were inserted in the database

* feat: merge disjoint ranges if possible before splitting

* fix: perf feedbacks

---------

Co-authored-by: leruaa <aurelien.catinon@gmail.com>

* chore: bump kona for Fusaka readiness (#633)

* kona bump wip

* patch kona

* compiles

* fmt

* revert

* chore(kona): bump version

* finally works

* chore: missing changes from merge commit

* bump to tag

* bump elfs

* fix l1 config path

* feat: support load l1 chain config from fs

* fix: cycle count on a range from 1 week ago

* revert: cycle count on a range from 1 week ago

This reverts commit ea93264558adcd43909fa42d73ca7ec3c398c7c2.

* chore: bump hana (#640)

* bump hana

* chore: bump version

* bump elfs

* bump hokulea

* fix

* test: mine blocks every 1 second

* switch e2e test ci to use mainnet endpoints

* revert

* fix

* fix

---------

Co-authored-by: theochap <theodore.chap@gmail.com>
Co-authored-by: leruaa <aurelien.catinon@gmail.com>

* fix: add timeouts to network prover calls to prevent deadlock (#639)

* fix: add timeouts to network prover calls to prevent deadlock

* refactor: improve

* chore: set NETWORK_CALL_TIMEOUT_SECS to 15s

* refactor: use cached dispute state for challenger (#629)

* fix(fault-proof): fix proposer with canonical chain tracking

* refactor: improve game resolution

* refactor: improve game credit claim

* refactor: improve game defense

* chore: add comments

* chore: rename to update_canonical_head_if_further

* chore: rename to is_XXX for boolean returning func

* refactor: remove early return from `refresh_state`

* fix

* fix

* refactor: refactor fix-proposer PR (#627)

* move run() to the top

* move state init from refresh state to proposer init

* huge refac

* remove is_finalized

* remove update_canonical_head_if_further

* move below new

* switch orders

* compute canonical head

* remove state snapshot

* typo

* refac remove subtree

* fmt

* update fetch_game

* rename to sync_state

* remove is_descendant_of_anchor

* refac remove_subtree

* more refac (#628)

* wip

* remove for claim_bond_for_game

* fix

* clippy

* fix

* fix

* fix

* clippy

* fix

* refac

* fix: drop if parent game is not in cache

* switch orders

* fix is_parent_resolved

* update testing doc

* remove cursor and set_cursor

* improve fetch_game

* improve sync_games

* misspell

* refac: cursor+deadline scan with gameType filter

- Remove MAX_GAMES_TO_CHECK_FOR_CHALLENGE
- Simplify malicious challenge

* move run to top

* wip

* huge refac + compiles

* more refac

* more refac

* more refac

* polish docs

* more refac

* udpate comment

* more refac

* update comments

* fix compiler error

* fix

* fix: validate cached checkpoints against contract before reuse (#641)

* fix: validate cached checkpoints against contract before reuse

* forge fmt for foundry 1.4.1-stable

* feat: add json logging (#643)

* feat: add JSON log format option to logger setup

* feat: add ansi feature flag and respect NO_COLOR in logger

* fix: clean up imports and formatting in logger.rs

* chore: clippy

---------

Co-authored-by: Victor Castell <0x@vcastellm.xyz>

* feat: add --slow for deploy to testnet or mainnet (#599)

Co-authored-by: Aurélien <3535019+leruaa@users.noreply.github.com>

* feat: add support for GCP HSM (#638)

* feat: add support for GCP HSM

* feat: use alloy-signer-gcp

* chore: add signer methods

* fix: signer test

* fix: typo

* feat: optional `L1_BEACON_RPC` (#645)

* feat: optional l1 beacon url

* nit

* refactor: return Result instead of Option

* fix: L1_BEACON_RPC in book

* nit

* fix: make fmt happy

* refactor: refactor run_range_program (#635)

* refactor: refactor run_range_program

* fix

* fmt

* chore: bump celestia range elf

* chore: bump elfs

* fix: return false for parent game challenger wins check when genesis (#650)

* fix: return false for parent game challenger wins check when genesis

* fix

---------

Co-authored-by: seolaoh <osa8361@gmail.com>

* feat: add a timeout for network calls (#646)

* fix: re-enable CI tests + resolve Anvil conflict + forge fmt (#660)

* fix: drop Anvil if test panics

* fix: validity docstring tests

* fix: forge fmt

* fix: ci.yaml + test-threads=1

* fix: forge fmt with nightly

* refactor: e2e feature flag + run from Justfile

* fix: bring back L1 RPCs for now

* nit: happy fmt

* fix: testing guide cmd

* fix: not skip test_honest_proposer/challenger

* docs: some notes and refinements for smoother onboarding (#662)

* docs: misc improvements primarily around fault-proofs

* fix: add just installation for testing

* nit

* refactor: improve fp upgrade process

* fix: mock verifier link

* fix: -v flag when docker down

* docs: easier validity quick start to follow

* fix: remove eigenda configs from env + note

* nit: consistent wording in expr sections

* fix: remove eigenda unnecessary variables

* feat: set the output path of the l2oo config (#652)

Allow to setup the output path of the config to an arbitrary file, this
is necessary when running the compiled version of fetch_l2oo_config as a
tool, for example using a Docker container.

* chore: bump kona to kona-client/v1.1.6 (#656)

* chore: bump kona to kona-client/v1.1.4

* fix

* wip

* fix

* chore: bump hana

* chore: bump hokulea

* chore: bump hokulea

* chore: bump hokulea

* fix bump hokulea

* bump elfs

* fix(fault-proof): clear stale canonical head value (#653)

* fix(fault-proof): clear stale canonical head value

* chore(fault-proof): add more logging

* fix(fault-proof): clear canonical_head_index only

canonical_head_l2_block serves as initial state

* test(fault-proof): add test_proposer_recovery_after_canonical_head_invalidation

* fmt

* test(fault-proof): increase timeout

* test(fault-proof): increase timeout

* test(fault-proof): increase timeout

* chore: bump kona to kona-client/v1.1.4

* fix

* wip

* fix

* update test to finalize valid games to be evicted

* test(fault-proof): increase timeout

* chore: bump hana

* chore: bump hokulea

* increase timeout

* chore: bump hokulea

* chore: bump hokulea

* fix bump hokulea

* bump elfs

* bump elfs

* fix(utils): add mutex to signer

* chore: remove unused import

* refactor: decouple locking from Signer enum via SignerLock wrapper

* refactor(proposer): remove redundant game creation decision

should_create_game(), now returns the parent game index alongside
the decision boolean and next block number.

* feat: make agg proof mode configurable for proposers (#671)

* feat: make agg proof mode configurable for fp proposer

* refactor: default to plonk for agg proofs

This is because plonk has better trust assumptions than groth16.

* make fmt + clippy happy

* fix(fault-proof): skip legacy games (#677)

* refactor: enrich MockPermissionedDisputeGame

* fix: e2e legacy games should use real claims

* refactor: separate drop games check + improved logging

* fix: skip legacy games

* fix: pass factory address for mock game deployment

* fix: simplify legacy game e2e test

* test: ensure proposer can handle game transition while running

* fix: make clippy happy

* docs: update game type transition e2e

* fix: increment cursor if game type not supported

* nit: redundant factory + typo

* fix(fault-proof): retain anchor game in cache (#676)

* fix(fault-proof): retain anchor game in cache

* refactor(fault-proof): fetch anchor outside state lock

* test(fault-proof): add proposer_retains_anchor_after_bond_claim

* ci: add fp integrations test workflow

* docs: update testing.md

* chore: make clippy happy

* test: move integration test to e2e suite

* refactor: enforce game retention policy at action creation

* chore: fmt

* chore: revert change in just tests

* feat: add sp1Verifier() getter to OPSuccinctFaultDisputeGame (#674)

* chore: improve fault proof contracts upgrade flow (#673)

* chore: remove private key for dry run

Also added private key validation on upgrade execution.

* docs(fault_proofs): add missing requirements

Added l1 rpc and private key required for the fp contracts upgrade
call.

* fix(fault-proof): derive proving interval from contract (#679)

* fix: compute witness interval from chain

* fix: less contract call

* fix: saturating sub

* fix: start_block

* refactor: filter e2e logging targets (#682)

* refactor: filter e2e logging targets

* fix: remove init_logging inside tests

* refactor: standalone init_logging

* fix: missing e2e

* fix: op_succinct_client_utils

Co-authored-by: Taehoon Kim <taehoon@succinct.xyz>

---------

Co-authored-by: Taehoon Kim <taehoon@succinct.xyz>

* fix: avoid full replay on restarts by backward sync (#681)

* fix: sync backwards to avoid caching from genesis

* refactor: define GameFetchResult

* fix: if syncing failed retry before continue

* fix: if cursor lt latest_index

* fix: define start_cursor

* chore: add some logs

* fix: check if game already exists

* test: e2e harness overhaul expanding `TestEnvironment` (#683)

* refactor: helper methods on TestEnvironment

* refactor: validation_anchor_reset

* refactor: send_contract_transaction

* chore: simplify e2e tests

* feat: define TestPrivateKeys

* refactor: game_type field

* refactor: monitoring methods on TestEnvironment

* refactor: deploy mock permissioned game

* nit: rename latest_game_info

* fix: remove const keys, type

* feat: allow config bins to load `.env` from a specified path (#667)

* feat: load .env from the specified path before falling back to root

* refactor: Try loading .env when project root missing

If project root isn't found, still attempt to load the provided env file
before emitting a warning to avoid spurious messages when the file
exists.

---------

Co-authored-by: Victor Castell <0x@vcastellm.xyz>

* fix: adjust action triggers + concurrency (#684)

* fix: action triggers + concurrency

* nit

* fix: apply review comments

* fix: preserve lowest cached game subtree + `sync_state` tests (#686)

* refactor: define init_proposer method

* test: some sync_state test cases

* refactor: add canonical_head_l2_block to snapshot

* test: complete cases for in_progress games

* deps: rstest

* refactor: extend to cover cases with anchor game

* chore: replace under e2e feature

* test: some anch cases

* test: test_sync_state_with_max_game_deadline_gap

* fix: do not remove subtree of lowest cached game

* refactor: define Assertion trait

* fix: update start_cursor only if fetching succeed

* test: test_sync_state_with_invalid_game_removes_subtree

* test: expand test cases for invalid games

* nit: some clean-ups

* ci: e2e timeout set to 60mins

* nit: typo

* ci: test target matrix

* fix: typo

* refactor: define Cursor type

* refactor: simplify invalid game eviction

* chore: remove unused Cursor methods

* nit: turn on e2e feature

* chore: bump kona to kona-client/v1.2.2 (#680)

* chore: bump kona to kona-client/v1.2.2

* feat: use a custom crypto for kzg point eval precompile

* feat: add cycle tracker back

* chore: bump elfs

* chore: fix fmt

* chore: bump hokulea

* chore: bump hana to v1.4.0-mocha

* chore: bump hokulea to hokulea-client/v1.1.0

* chore: bump to v3.4.0-rc.1 (#692)

* chore: bump to v3.4.0

* chore: bump to 3.4.0-rc.1

* fix: use `RwLock` for state + lower `fetch_interval` for tests (#691)

* fix: switch to RwLock for state

* fix: lower fetch_interval for tests

* fix: try with fetch_interval=5

* fix: test_proposer_recovery_after_canonical_head_invalidation

* nit

* fix: missing e2e feature flag

* fix: up to 10

* fix: look up for correct parent_id

* nit: bring back e2e feature

* nit: docstring

* fix: single lock on removing subtree of invalid games

* refactor(fault-proof): avoid IO under write lock (#693)

* fix: switch to RwLock for state

* fix: lower fetch_interval for tests

* fix: try with fetch_interval=5

* fix: test_proposer_recovery_after_canonical_head_invalidation

* nit

* fix: missing e2e feature flag

* fix: up to 10

* fix: look up for correct parent_id

* nit: bring back e2e feature

* nit: docstring

* fix: single lock on removing subtree of invalid games

* refactor(fault-proof): avoid IO under write lock

---------

Co-authored-by: Farhad Shabani <farhad.shabani@gmail.com>

* fix: update Cargo.lock (#696)

* fix: update Cargo.lock

* chore: update elf

* ci: add `just update-elf` (#697)

* fix: update Cargo.lock

* chore: update elf

* fix: higher timeout for *_proposer_running

* feat: add just update-elf

* refactor: use just in elf CI

* nit: rm newline

* fix: breakdown elf just cmds + update verify-binaries.md

* nit

* fix: revert timeouts

---------

Co-authored-by: Taehoon Kim <taehoon@succinct.xyz>

* feat: allow custom directory for L1/L2 config files (#699)

* feat: accept custom L1/L2 config dirs

* refactor: setup logger and add logs

* doc: add L1/L2 config dir under optional vars

* nit: consistent log message

* fix: multilingual removed

* fix: mdbook lints

* fix: keep mdbook on v0.4 for now

* fix: mdbook-mermaid@0.16.0

* fix: default values in book

* feat: log config from file if exists

* docs: fix fault-proof proposer.md (#672)

* docs: fix typo in proposer env file

* docs: remove unnecessary direction

* docs: fix typo

* docs: align challenger.md

* test(fault-proof): add integrations test for unsupported/invalid games (#694)

* test(fault-proof): add test_invalid_game_type_handling

* test(fault-proof): remove redundant legacy game filtering e2e test

* test: add missing import

* test(fault-proof): add test_sync_state_filters_multiple_legacy_games

* test(fault-proof): remove redundant legacy game filter e2e test

* test(fault-proof): add test_sync_state_filters_non_respected_game_type

* test(fault-proof): restore game type transition e2e tests

* test(fault-proof): consolidate redundant legacy game filtering tests

* docs: update comment for test_sync_state_filters_legacy_games

* test: add test helpers and restore e2e tests

* nit: make fmt happy

* nit: make clippy happy

---------

Co-authored-by: Farhad Shabani <farhad.shabani@gmail.com>

* chore: bump hokulea to hokulea-client/v1.1.1 (#703)

* test: sysgo-powered e2e testing infra for validity proposer (#701)

* fix: higher timeout for *_proposer_running

* feat: add just build-elfs

* nit: rm newline

* feat: setup minimal e2e test with op-devstack

* feat: Justfile

* feat: binding generator script

* test: get minimal test working

* fix: update-package

* refactor: gen few bindings and use for init_test

* chore: apply changes from /optimism

* chore: revert irrelavant changes

* test: TestValidityProposer_L2OODeployedAndUp

* chore: rename package to validity_test

* deps: redirect to succinctlabs/optimism

* refactor: override some params

* chore: separate out presets from tests

* feat: deploy SP1MockVerifier after

* fix: do not require ETHERSCAN_API_KEY

* ci: e2e-sysgo.yml

* fix: missing working-directory

* chore: separate out adapter code

* fix: trim just cmds

* refactor: supervisor wirings

* test: TestValidityProposer_ProvingSingleRange

* deps: update

* feat: NextBlockNumber

* fix: start proving from 1 + default sequencing window

* fix: latest L2 block number 1 _L2OODeployedAndUp

* docs: add README

* chore: replace gen-binding.sh

* deps: update optimism rev

* ci: switch to runs-on

* chore: rm unnecessary compressed/README

* fix: submodule

* docs: Maintenance section

* deps: latest rev on op-succinct-sysgo

* fix: WithL2OOStartingBlockNumber

* feat: add preflight check script for fp (#669)

* feat: add preflight check script for fp

* docs: update and fix

* chore: cleanup

* chore: default to .env.preflight

* chore: apply comments

* docs: add note about altda feature flags

* feat: introduce SET_IMPL_BLOCK env var

* refactor: make AGG_PROOF_MODE configurable

* chore: make USE_KMS_REQUESTER optional

* docs: make l1 beacon rpc required

* configurable CHUNK_SIZE

* fix: effective strategy

* fix

* cleanup

* docs

* docs

---------

Co-authored-by: Farhad Shabani <Farhad.Shabani@gmail.com>

* test(fault-proof): add bond claim eviction test (#698)

* test(fault-proof): add bond-claim eviction test

* test(fault-proof): remove unnecessary step

* test(fault-proof): add test_bond_claim_eviction_multi_branch

* chore: gitignore foundry.lock (#709)

* test(fault-proof): add test_challenger_wins_cascade_removal (#700)

* test(fault-proof): add test_invalid_game_type_handling

* test(fault-proof): remove redundant legacy game filtering e2e test

* test: add missing import

* test(fault-proof): add test_sync_state_filters_multiple_legacy_games

* test(fault-proof): remove redundant legacy game filter e2e test

* test(fault-proof): add test_sync_state_filters_non_respected_game_type

* test(fault-proof): restore game type transition e2e tests

* test(fault-proof): consolidate redundant legacy game filtering tests

* docs: update comment for test_sync_state_filters_legacy_games

* test: add test helpers and restore e2e tests

* test(fault-proof): add bond-claim eviction test

* test(fault-proof): add test_challenger_wins_cascade_removal

* test: use new methods

* chore: fmt

* chore: typo

* test(fault-proof): add test_mixed_states_multiple_branches

* test(fault-proof): add cases with anchor to cascade removal

* test(fault-proof): fix to challenge before game is over

* test(fault-proof): add multiple challenger wins branches case

* check if evicted from cache

* test(fault-proof): add anchor_parallel_challenger_wins_with_child

* typo

* test: add proposer default fault_dispute_game helper

* test(fault-proof): add comprehensive game state marking tests (#702)

* test(fault-proof): add test_invalid_game_type_handling

* test(fault-proof): remove redundant legacy game filtering e2e test

* test: add missing import

* test(fault-proof): add test_sync_state_filters_multiple_legacy_games

* test(fault-proof): remove redundant legacy game filter e2e test

* test(fault-proof): add test_sync_state_filters_non_respected_game_type

* test(fault-proof): restore game type transition e2e tests

* test(fault-proof): consolidate redundant legacy game filtering tests

* docs: update comment for test_sync_state_filters_legacy_games

* test: add test helpers and restore e2e tests

* test(fault-proof): add bond-claim eviction test

* test(fault-proof): add test_challenger_wins_cascade_removal

* test: use new methods

* chore: fmt

* chore: typo

* test(fault-proof): add test_mixed_states_multiple_branches

* test(fault-proof): add cases with anchor to cascade removal

* test(fault-proof): fix to challenge before game is over

* test(fault-proof): add test_in_progress_games_resolution_marking

* make clippy happy

* test(fault-proof): add test_bond_claim_marking

* test(fault-proof): add proof provided cases

* surface errors via ?

* drop test doc to reflect current coverage

* use default fault dispute game accessor

* drop stale changes in bond claim eviction test

* chore: remove kurtosis (#711)

* fix: fail to create config when finalized L2 block is below the finality window (#705)

* fix: error if not enough finalized L2 block

* chore: same logic for fdg config

* fix: no need for error log

* doc: add note for why excluded genesis block

* nit

* test(fault-proof): add more targeted integrations tests (#706)

* test(fault-proof): add test_invalid_game_type_handling

* test(fault-proof): remove redundant legacy game filtering e2e test

* test: add missing import

* test(fault-proof): add test_sync_state_filters_multiple_legacy_games

* test(fault-proof): remove redundant legacy game filter e2e test

* test(fault-proof): add test_sync_state_filters_non_respected_game_type

* test(fault-proof): restore game type transition e2e tests

* test(fault-proof): consolidate redundant legacy game filtering tests

* docs: update comment for test_sync_state_filters_legacy_games

* test: add test helpers and restore e2e tests

* test(fault-proof): add bond-claim eviction test

* test(fault-proof): add test_challenger_wins_cascade_removal

* test: use new methods

* chore: fmt

* chore: typo

* test(fault-proof): add test_mixed_states_multiple_branches

* test(fault-proof): add cases with anchor to cascade removal

* test(fault-proof): fix to challenge before game is over

* test(fault-proof): add test_in_progress_games_resolution_marking

* make clippy happy

* test(fault-proof): add test_bond_claim_marking

* test(fault-proof): add proof provided cases

* test(fault-proof): add more targeted integrations tests

* test(fault-proof): add test for mixed proposal statuses across branches

* test(fault-proof): add test_three_sibling_states_eviction_and_pruning

* assert anchor game credit

* fix test_anchor_with_zero_credit_sibling_and_in_progress_branch

* clean up unintented merge

* feat(fault-proof): validate starting L2 block (#704)

* feat(fault-proof): validate starting L2 block

- Add check that contract anchor L2 block does not exceed current finalized L2 block
- Improve error messaging for misconfigured startingL2BlockNumber
- Add e2e test rejecting future starting blocks

* typo

* introduce validate_anchor_l2_block

* handle error

* choose +1 offset to test boundaries

* make fmt happy

* test: sysgo-powered e2e testing infra for fault-proof proposer (#708)

* feat: setup fault-proof e2e

* chore: reorg /e2e

* refactor: shared preset

* feat: dispute game factory binding

* refactor: resuable call function in adapters

* test: L2DfgDeployedAndUp

* ci: add faultproof mode

* test: detect first game created

* chore: gen binding for OPSuccinctFaultDisputeGame

* nit: naming

* fix: add FailNow if timeout passed

* refactor: benefit from ethCaller

* refactor: WaitForGameCount

* test: check first game's parentId

* test: verify rootClaim

* chore: define opts

* fix: update /optimism rev

* chore: update /optimism rev

* fix: require.Equal in _L2DgfDeployedAndUp

* fix: wait for finalized L2 block before running sysgo e2e tests (#712)

* fix: update optimism rev

* chore: update tests/optimism

* chore: bump to v3.4.0 (#710)

* chore: bump hokulea to hokulea-client/v1.1.2

* chore: bump to v3.4.0

* docs: add RELEASE.md (#717)

* chore: add just vkeys

* docs: add RELEASE.md

* docs: missing Cargo.lock

* docs: clarify vkeys output in release guide

* chore: small changes

* ci: use on-demand runners and increase disk for e2e workflows (#718)

* fix: use on-demand runners and increase disk

* fix: missing spot=false for cost estimator (1000 blocks)

* nit: rm whitespace

Co-authored-by: Taehoon Kim <taehoon@succinct.xyz>

---------

Co-authored-by: Taehoon Kim <taehoon@succinct.xyz>

* chore: reorg test structure and improve workflow naming (#719)

* refactor: update test-e2e-sysgo

* chore: reorg /tests/e2e

* chore: update README

* chore: rename fault-proof e2e feature to integration

* ci: separate out Cost Estimator into its own yml file

* ci: reflect e2e > integration

* nit: missing update to integration

* nit: better jobs name

* chore: rename ci.yml to cargo-tests.yaml

* chore: e2e-sysgo > e2e-tests

* chore: foundry_test > foundry-test

* chore: merge cost estimator jobs

* chore: rename pr.yaml to lint.yaml

* chore: consistent .yml extension + pr-title

* fix: revert tests/optimism

* refactor: actually separate cost estimator workflows

* chore: renaming to e2e-sysgo-tests

* feat(fault-proof): support split range proving (#715)

* feat: can split proving range to 2,4,8

* chore: rename to RangeSplits

* chore: better error handling in prove_game

* refactor: review split_range logic

* refactor: turn split_range to method on RangeSplits

* test: add unit tests for split()

* refactor: use slice instead of vec in tests

* refactor: allow continuous range 1 to 16

* doc: docstring for RangeSplitCount methods

* chore: some logs

* feat: add Status + WaitForDefenderWins

* feat: add FastFinality preset

* test: proving tests

* chore: rename /proving to /fastfinality

* chore: rename faultproof_test to bootstrap_test

* deps: update optimism rev

* test: RangeSplitCount=4

* feat: max_concurrent_range_proofs

* docs: note for max_concurrent_range_proofs

* refactor: set max concurrency to 16 fo tests

* chore: range/agg_proof_request

* docs: add to optional vars

* test: check 5th game instead

* ci: free disk space

* fix: set fast finality proving limit to 8

* fix: longer timeout for _WaitsForFifthGameDefenderWins

* refactor: define GameStatus

* fix: use NonZeroUsize for max_concurrent_range_proofs

* fix: clearer anyhow message if start < end

* doc: ceiling devision note

* fix: test_errors_on_reversed_bounds assertion

* fix: remove go test -timeout flag

* test: add parallel range proving tests

* fix: bring back go test timeout

* chore: tune params so run on CI

* feat: configurable faultproof test setup

* test: add _RangeSplit1

* nit: remove Proposer config tracing

* deps: update optimism rev

* chore: tweak name of tests

* fix: only ThreeGames as more is resource exhausting

* docs: clarify SP1 mock verifier auto-deployment behavior (#721)

* fix(l2oo): validate submission interval is non-zero in `updateSubmissionInterval` (#724)

* fix: prevent update submission interval to zero

* test: _CannotSettoZero

* test: extend validity proposer "happy-path" e2e tests (#726)

* refactor: make validity e2e tests configurable

* feat: define OutputProposal + WaitForLatestBlockNumber

* test: add proving_test

* fix: rm now redundant _ProveSingleRange test

* chore: reorg adapters.go

* chore: verify L2 block number aligns

* feat: create ValiditySystem wrapper with DatabaseURL()

* feat: define Database client + verifyRange

* test: some tests for block-based splitting

* deps: update tests/optimism

* feat: configurable intervals for l2oo deployment

* ci: more parallel e2e jobs

* test: _RangeIntervalLargerThanSubmission

* feat: enable parallel execution

* make vars automatically exported for fdg-contracts deployment

* fix: consolidate config paths

* doc: comment for per-test isolated system

* deps: point to op-succinct-sysgo rev

* fix: forge fmt

* fix: smt went wrong with last fmt check!

* test: restart recovery harness and e2e tests (#728)

* feat: can restart Validity proposer

* test: Validity proposer restart recovery test

* test: fp proposer recovery tests

* ci: add recovery test jobs

* test: more validity recovery tests (ThreeSubmissions + RangeSplit)

* test: Validity multiple restarts

* refactor: reusable funcs instead of runRecoveryTest

* test: Fault Proof multiple restarts

* chore: rename from _Optimistic to _Basic

* refactor: add client factory methods to System types

* fix: forge install iff not already present

* chore: for FP restart no wait time

* refactor: tighter loopInterval + better doc around

* fix: make fastfinality tests parallel

* fix: rm dup ParallelT

* fix: game status require message

* fix: make loopInterval optional

* deps: update optimism rev

* test(fault-proof): clarify range splitting behavior once interval drifts (#734)

* test: cover interval drift for range splitting

* nit: range sizes

* nit

* doc: range proving semantic

* nit: happy cargo fmt

* test(eigenda): host tests + fault-proof CI coverage (#730)

* test: EigenDA host integration tests

* test: witness_generator unit/integration tests

* test: fp integration tests with eigenda feature enabled

* ci: add da-host-tests for eigenda

* chore: improve just commands

* fix: SRS path

* fix: no need for EigenDA SRS

* fix: enable mock mode

* fix(temp): verify EigenDA SRS file

* fix: create symlink for SRS file

* ci: magic cache with runs-on/action

* fix: $GITHUB_WORKSPACE absolute paths + existence check

* refactor: verify canoe proof deser when present

* refactor: Justfile handles symlink for both local and CI

* fix: L2_BLOCK_OFFSET_FROM_FINALIZED=200

* feat: log proposer/challenger configs on loads (#722)

* feat: add structured logging for proposer/challenger configs

* fix: add op_succinct_config_name_hash

* feat: log ProgramConfig

* fix(validity): correct gap detection in agg proof validation (#729)

* fix: reorder overlap/gap checks in agg validation

* chore: reorder validations in docstring

* fix: check < for gap

* fix(validity): rm redundant sort and add db client tests (#733)

* feat: db/client testing harness + few tests

* test: chain_lock + fetch_completed_ranges module

* chore: update Cargo.lock

* test: test_get_consecutive_***_ordered

* test: fetch_active_agg excludes failed & cancelled

* test: test insert requests chunking logic

* test: test_fetch_first_unrequested_range_proof_returns_lowest_start_block

* test: test_queries_filter_by_commitment_config

* fix: all active status should be counted

* refactor: rm db.cleanup().await

* fix: rm redundant sort

* chore: reorg

* chore: cargo fmt

* fix: mut keyword is now unused

* chore: L1ID / L2ID

* chore: rm redundant comments

* test: test_queries_isolate_by_chain_id

* test: coverage for identified gaps

* nit: cleanup unused variable

* refactor: define count() helper

* nit

* fix(fault-proof): conditionally allow earlier-parent games as canonical head (#736)

* fix: allow games with parent < anchor become canonical

* test: canonical head override logic

* fix: merge cases into test_sync_state_happy_paths

* fix: update compute_canonical_head comment

* chore: bump sp1 to v5.2.4 (#739)

* chore: release v3.4.1 (#740)

* chore: release v3.4.1

* ci: do not run on pushes to release/*

---------

Co-authored-by: Farhad Shabani <farhad.shabani@gmail.com>

* fix: propagate error instead of unwrap across pipeline (#713)

* fix: propogate instead of unwrap()

* fix: handle invalid preimage key

* chore: clean-ups

* chore: error log if witness generation failed

* fix: rm assert_eq

* chore: consistent log/error message

* fix: handle lock error

* fix: fmt happy

* fix: update ELF files

* test: long-running e2e tests with real proving + monitoring dashboards (#731)

* feat: can write env files

* test: long running Validity test

* test: long running fault-proof test

* feat: add long-running just commands

* chore: add README section

* refactor: graceful shutdown

* chore: rm redundant code

* fix: dropped configs in fp presets

* refactor: fp _LongRunning vs _Progress test

* refactor: validity _LongRunning vs _Progress test

* chore: update Justfile

* chore: split into progress and running test

* ci: add progress tests

* chore: rename to continuous_test

* feat: define LongRunningValidityConfig

* refactor: LongRunningFaultProofConfig

* refactor: set envFilePath only for continuous_test

* refactor: tune fp params for progress_tests

* test: TestFaultProofProposer_FastFinality_Progress

* refactor: tune Validity progress_test

* refactor: RunForDuration - 15mins

* chore: extract runners to separate file

* ci: PROGRESS_TEST_DURATION=30m for main

* fix: more concurrent range proofs for Validity

* fix: increase concurrency for fp

* fix: RangeSplitCount=2

* fix: consistent continuous_tests

* fix: tune some params

* feat: add monitoring for validity

* feat: long running fast finality

* refactor: dedicated fast finality config func

* feat: faultproof monitoring dashboard

* test: in ff check at least one game proven

* chore: better /monitoring org + set data dir

* refactor: make l2BlockTime configurable

* chore: misc

* feat: anchor state registry bindings

* feat: checkAnchorStateLag

* chore: re-tune since now l2BlockTime=2s

* ci: overhaul e2e-sysgo-tests.yml

* fix: higher intervals b/c of real proving

* fix: do no FAIL in _LongRunning

* fix: keep cmd foreground so ctrl+c works

* chore: rebase issues

* fix: propogate LatestBlockNumber err

* refactor: output root verification + simplify lag checks

* refactor: larger intervals for real proving

* fix: rm runs-on cache

* fix: broken Celestia light-node link

* fix: l2BlockTime default to 1s

* ci: separate network proving workflow file

* fix: set batcher maxBlock.. + maxChannel..

* chore: re-tune b/c of larger batching size

* feat: set proving timeout

* feat: use real sp1 verifier

* feat: AggProofMode

* deps: update tests/optimism rev

* ci: build contracts

* chore: update README

* fix: check if symlinks exist

* fix: consistent test timeout

* fix: monitoring thresholds

* fix: unused time dep

* fix: MaxConcurrent*** optional

* fix: make contracts build without test submodules

* fix: get l2BlockTime from chain

* refactor: define L2ChainConfig

* refactor: add ProposerOptions

* fix(fault-proof): sort defense candidates by deadline ascending (#737)

* fix(fault-proof): sort defense candidates by deadline ascending

* test(fault-proof): verify defense tasks prioritize earliest deadline

* fix(l2oo): prevent challenger from deleting genesis output (#723)

* fix: disallow challenger to delete genesis

* test: Cannot/CanDeleteGenesisOutput

* chore: comment genesis cannot be deleted

* feat(tests): running standalone L1/L2 devnet nodes (#743)

* feat: TestNodes_RunOnly

* refactor: logged just nodes

* refactor: simplify configs storage

* docs: update tests/README

* deps: update tests/optimism

* refactor: merge nodes mode command into long-running

* docs: better README structure

* chore: rename to continuous_test

* feat: use FinalizationPeriodSecs

* deps: update tests/optimism rev

* nit: whitespace

* fix: rm redundant fetcher instance (#744)

* fix(fault-proof): check respected game type before creating games (#746)

* fix: disallow game creation if not respected

* docs: add PORTAL_ADDRESS

* deps: update tests/optimism

* refactor: take protal for propoer::new()

* fix: cargo fmt

* refactor: improve log

* refactor: switch to ANCHOR_STATE_REGISTRY_ADDRESS

* fix: higher l2 block offset b/c of EigenDA

* fix: read from proposer_config

* fix: clippy

* fix(fault-proof): proper pre-migration startup + consolidate anchorStateRegistry RPC calls (#748)

* fix: validate anchor_state_registry + define startup validations

* refactor: challenger startup validaitons

* fix: update preflight

* docs: add ANCHOR_STATE_REGISTRY_ADDRESS

* fix: l2 anchor validation tests

* refactor: explicit semantic with OnceLock<U256>

* refactor: decouple startup validation from state init

* chore: cleanups

* chore: reorder

* fix: happy fmt

* refactor: define try_init

* refactor: move proposer init_bond after validation

* fix: test_proposer_rejects_future_starting_block

* fix: test_proposer_retains_anchor_after_bond_claim

* refactor: rm proposer var

* docs: bring back loop's comment

* fix: propogate errors

* fix: factory typo

* docs: improve quick-start guides (#754)

* docs: improve quick-start guides

* fix: ubuntu deps note for fp

* fix(fault-proof): bound gas for getLastProposalTimestamp call during migration (#755)

* fix: AccessManager.sol should not burn all gas to run getLastProposalTimestamp

* test: getLastProposerTimestamp

* fix: forge fmt

* fix: lt

---------

Co-authored-by: Kien Nguyen <kien@riselabs.xyz>

* fix(tests): higher integration tests timeout (#742)

* fix: higher integration tests timeout

* refactor: define WAIT_TIMEOUT

* fix: better logs for GameCount

* fix: typos

* fix: 10 blocks ahead for checking future statring block

* refactor: set WAIT_TIMEOUT=60

* feat(fault-proof): custom proof polling with retries, timeouts, and failure metrics (#745)

* feat: NETWORK_CALL_TIMEOUT + AUCTION_TIMEOUT

* refactor: decouple submission from completion

* fix: AUCTION_TIMEOUT=60s

* fix: happy fmt

* refactor: NETWORK_CALLS_TIMEOUT=15s

* fix: log cancel on network

* fix: happy fmt

* refactor: define prover mod

* refactor: ProposerConfig

* chore: update test config

* test: timeout unit tests

* chore: cleanups

* chore: add timeout metrics

* nit: typo

* fix: skip proving if deadline passed

* test: proving deadline unit tests

* chore: reorg network call method

* refactor: misc improvements

* fix: rm agg_vk

* chore: sdk's polling interval permalink

* fix: rename to should_skip_proving

* fix: rebasing issues

* fix: add ProposerGauge::DeadlineApproaching

* fix(fault-proof): add transaction receipt status checks (#751)

* fix: add tx status checks

* fix: improve tx error handling + logging

* fix: define TX_REVERTED_PREFIX

* feat(tests): add challenger support to faultproof sysgo infra  (#757)

* test: fp challenger bootstrap test

* fix: consistent naming for fp configs

* refactor: run challenger for long-running tests

* fix: update Justfile

* chore: better faultproof.go organization

* fix: set challenger env file path

* chore: update faultproof grafana dashboard

* fix: explicitly pass WithDefaultChallenger()

* fix: check nil in challenger Start/Stop

* fix: misc touchups

* refactor: reusable newSystemWithProposer

* chore: update tests/optimism rev

* add test coverage (#762)

Co-authored-by: TiesD <71668189+TiesD@users.noreply.github.com>

* fix(fault-proof): challenger `sync_state` update and deadline check (#759)

* fix: challenger sync bugs

* test: cover fixes

* nit

* fix: test_proposer_rejects_future_starting_block

* nit: 100 block beyond

Co-authored-by: Taehoon Kim <taehoon@succinct.xyz>

---------

Co-authored-by: Taehoon Kim <taehoon@succinct.xyz>

* feat(fault-proof): proposer state backup for faster recovery (#756)

* feat: backup fp cache

* docs: update proposer.md

* test: unit and integration backup tests

* fix: dedicated blocking thread pool

* fix: test_proposer_backup_persistence

* test: add recovery test with backup

* fix: rebasing issue

* nit: touchups

* chore: rename to BACKUP_PATH

* chore: address comments (#763)

* chore(fault-proof): add comment explaining non-persisted backup fields

* fix(fault-proof): use tempfile for atomic backup writes

* fix(fault-proof): validate backup path at startup

* feat(fault-proof): add backup metrics for observability

* refactor(fault-proof): improve backup path validation and restore logic

Consolidate duplicate if-let blocks, add tracing for restore operations,
handle empty parent path edge case for relative paths, and log cleanup failures.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* test(fault-proof): add schema guard test for backup version

Add a test that fails if Game or ProposerBackup fields change,
reminding developers to bump BACKUP_VERSION when schema changes.
This catches accidental schema changes that would break backup
compatibility.

* refactor(fault-proof): simplify backup restore logic

Remove redundant path.exists() check before try_restore. Now only logs
error metric when file exists but couldn't be parsed, avoiding double
filesystem access and correctly distinguishing missing file from corrupt.

* refactor(fault-proof): use tempfile for backup path write test

Replace manual File::create + remove_file with NamedTempFile for the
write permission check. Ensures cleanup on panic and is consistent
with the atomic write approach used in backup.rs.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* refactor(fault-proof): remove duplicate backup restore log

The log in ProposerState::try_restore() already provides restore
information with game count and cursor details.

* fix(tests): update optimism submodule for backup support

The submodule was reverted to an older commit during a merge with main.
This restores the correct commit that includes WithFPBackupPath needed
by the faultproof test presets.

* fix(tests): update optimism submodule to merged backup+challenger

Updates tests/optimism to 32bd7c930 which includes PR #325 merge
of backup support into op-succinct-sysgo branch.

---------

Co-authored-by: Taehoon Kim <taehoon@succinct.xyz>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* test(fault-proof): challenger `sync_state` coverage (#760)

* test: challenger sync_state coverage

* test: add parent resolved resolution marking test

Add test_resolution_marking_parent_resolved to complement the existing
test_resolution_marking_parent_not_resolved, fully covering the
is_parent_resolved check logic in sync_state.

* fix(test): resolve timing issue in parent_challenger_wins test

Create child game after parent is resolved to avoid challenge window
expiration. The 12-hour time warp (MAX_PROVE_DURATION) was exceeding
the 1-hour challenge window (MAX_CHALLENGE_DURATION) for pre-created
child games.

* refactor(test): simplify test_game_eviction parameters

Remove redundant expected_evicted parameter that was always true.

* style: cargo fmt

* fix(test): use two-stage time warp for parent_challenger_wins test

Create child game DURING the prove period (not after resolution) to
avoid contract rejection of children of defeated games, while ensuring
child's challenge window is still open when parent resolves.

---------

Co-authored-by: fakedev9999 <taehoon@succinct.xyz>

* fix(logger): use JsonFields for proper span serialization in JSON logs (#761)

This fixes the `field_error: "EOF while parsing a value"` issue that
occurs when using JSON logging format (LOG_FORMAT=json).

Root cause: When using `.event_format(Format::default().json())` without
also setting `.fmt_fields(JsonFields::new())`, the layer uses DefaultFields
(text formatter) instead of JsonFields. This mismatch causes the JSON
serializer to fail when parsing the text-formatted span fields.

This is a known issue: https://github.com/tokio-rs/tracing/issues/1365

The fix properly configures both the event format AND field format to use
JSON, which preserves span context in logs (useful for debugging and
distributed tracing) rather than removing it entirely.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* feat(tests): add e2e test for proposer defense after challenge (#764)

* feat(tests): add e2e test for proposer defense after challenge

Add a new e2e test that verifies the complete defense flow when a
proposer's valid game is challenged and the proposer successfully
defends by submitting a proof.

Test scenario:
1. Proposer creates a valid game
2. Malicious challenger challenges the game (100% challenge rate)
3. Proposer detects challenge and generates proof
4. Proposer submits proof on-chain
5. Game resolves as DEFENDER_WINS

Changes:
- Add ProposalStatus type and WaitForProposalStatus helper to adapters.go
- Create new defense test in tests/e2e/faultproof/defense/
- Add defense tests to CI workflow matrix

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(tests): add String() method to ProposalStatus for log readability

* chore: update tests/optimism submodule for float64 fix

Updates the optimism submodule to include the fix for float64
formatting in setEnvIfNotNil. This resolves the challenger crash
when using MaliciousChallengePercentage config.

Required for: defense e2e test

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(tests): increase MaxProveDuration for defense test

The default 10s prove duration was too short for the proposer to
respond to challenges in CI. Increased to 60s to allow time for
detecting the challenge, generating the mock proof, and submitting
the transaction.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* fix(scripts): use CLI --default-range argument instead of hardcoded constant (#773)

* fix(scripts): use CLI --default-range argument instead of hardcoded constant

The --default-range CLI argument was being parsed but its value was never
used. The hardcoded DEFAULT_RANGE constant was passed to get_validated_block_range
instead of the user-provided args.default_range value.

* refactor(scripts): move DEFAULT_RANGE constant to test common module

Since DEFAULT_RANGE is now only used in tests, move it from the public
library to the test common module as suggested by @Farhad-Shabani.

* fix: use bytecode_address instead of target_address for precompile lookup

* fix(ci): prevent duplicate misspell check run in GitHub UI (#766)

The reviewdog/action-misspell action was creating a separate check run
via the GitHub Checks API that got misassociated with other workflows
(appearing as "Build OP Succinct Lite Docker Images / misspell").

Changes:
- Switch reporter from github-pr-check to github-pr-review
- Use lowercase github_token (correct input name)

This eliminates the redundant check run while providing more useful
inline PR review comments for spelling errors.

* chore: upgrade to op-contracts v5.0.0 (#774)

* chore: upgrade Conduit to op-contracts v5.0.0

Update op-contracts submodule to v5.0.0 and refactor code to support breaking changes:

**Solidity Changes:**
- Rename OutputRoot → Proposal and l2BlockNumber → l2SequenceNumber across all contracts
- Update AnchorStateRegistry initialization to use new v5.0.0 signature with ISystemConfig
- Switch from ERC1967Proxy to Optimism Proxy + ProxyAdmin for contracts using ProxyAdminOwnedBase
- Update test setup to use new proxy pattern and guardian-based blacklist/retirement mechanisms
- Add MockSystemConfig contract to support guardian-restricted operations in tests

**Rust Changes:**
- Update contract.rs ABI definitions to use l2SequenceNumber instead of l2BlockNumber
- Fix Rust function calls in proposer.rs, challenger.rs, and test files
- Update test utilities to use deployed contract addresses instead of AnchorStateRegistry methods
- Fix DisputeGameFactory function selector disambiguation for overloaded setImplementation

**Testing:**
- All Solidity tests pass (54/55, 1 skipped due to env var requirement)
- All Rust tests pass (58 unit + integration tests)
- Updated testParentGameNotValid to use AnchorStateRegistry blacklist
- Renamed testParentGameNotRespected to testParentGameRetired to reflect new mechanism

* fix(test): update set_respected_game_type to use AnchorStateRegistry

In op-contracts v5.0.0, respectedGameType is stored in AnchorStateRegistry
instead of OptimismPortal2. Update the test environment to call
AnchorStateRegistry.setRespectedGameType() to properly set the respected
game type for integration tests.

This fixes test_sync_state_filters_non_respected_game_type tests which
verify games created with non-respected types have wasRespectedGameTypeWhenCreated=false.

* fix(fp): add l2BlockNumber() alias for backward compatibility

The test bindings call l2BlockNumber() but OPSuccinctFaultDisputeGame
only had l2SequenceNumber(). This caused E2E test failures with
"call l2BlockNumber: execution reverted: 0x".

Add l2BlockNumber() as an alias that delegates to l2SequenceNumber()
to maintain compatibility with the test infrastructure.

* feat(validity): add anchorStateRegistry() for v5.0.0 compatibility

The v5.0.0 AnchorStateRegistry.isGameRegistered() calls anchorStateRegistry()
on all dispute games during withdrawal proof validation. This adds the required
function to OPSuccinctDisputeGame (validity mode) to prevent withdrawal failures.

Changes:
- Add IAnchorStateRegistry import and immutable variable
- Update constructor to accept anchor state registry parameter
- Add anchorStateRegistry() getter function
- Update deployment script to read ANCHOR_STATE_REGISTRY from env
- Update tests with anchor state registry setup

* style: fix forge fmt formatting in validity tests

* fix: use the correct l2_chain_id (#782)

Co-authored-by: Kien Nguyen <kien@riselabs.xyz>

* feat: add SystemConfig support for AnchorStateRegistry (#779)

* feat(fdg): add SystemConfig support for AnchorStateRegistry

- Add systemConfigAddress field to FDGConfig struct
- Auto-derive SystemConfig address from rollup config
- Support zero address to deploy MockSystemConfig for testing
- Use existing SystemConfig for production deployments
- Refactor rollup_config extraction to eliminate redundancy

* fix(tests): add missing system_config_address field in test config

* style: fix rustfmt formatting in fetch_fault_dispute_game_config

* chore: align default verifier to Plonk VerifierGateway (#725)

* feat: bump kona to v1.2.7, hokulea to v1.1.4, alloy to 1.4.x (#778)

* chore: bump kona to v1.2.7 and compatible dependencies

Upgrades:
- kona: v1.2.2 → v1.2.7 (13 crates)
- hokulea: v1.1.2 → v1.1.3 (9 crates)
- hana: v1.4.0-mocha → v1.5.0-mocha (4 crates)
- alloy versions: 1.1.0 → 1.1.3 (pinned to =1.1.3)
- tracing-subscriber: 0.3.20 → 0.3.22
- postgresql_embedded: 0.15 → 0.20 (fixes reqwest_middleware conflict)

Code changes for hokulea v1.1.3 API compatibility:
- Removed recency window support (removed from hokulea v1.1.3)
- Updated CanoeSp1CCReducedProofProvider to use RpcClient instead of URL string
- Changed oracle.clone() to oracle.as_ref() in from_boot_info_to_canoe_proof
- Added alloy-rpc-client dependency to eigenda host utils

Pinned vergen versions for rollup-boost compatibility.

Verified with: cargo clippy --all-features --lib --bins

* fix: use succinctlabs/reth fork for reth patches (CI compatibility)

The reth patches are needed to fix a PartialEq bug in reth-transaction-pool
(PR #21336). Cargo requires patches to come from a different source URL
than the original dependency.

Previously used SSH URLs (ssh://git@github.com/...) which don't work in CI
environments without SSH keys configured.

Now uses the succinctlabs/reth fork with the v1.9.3-kona branch, which:
- Contains the same fix as paradigmxyz/reth v1.9.3-kona branch
- Uses HTTPS URLs that work in CI
- Provides a different source URL to satisfy Cargo's patch requirements

* chore: cargo fmt

* chore: bump elfs

* feat: bump hokulea to v1.1.4 and alloy to 1.4.x

Upgrade hokulea from v1.1.3 to v1.1.4, which requires:
- Updating alloy versions from =1.1.3 to 1.4 for semver compatibility
- Removing the alloy-tx-macros crates.io patch (no longer needed)

hokulea v1.1.4 brings:
- Extracted canoe_proof_stdin, generate_canoe_proof functions
- Bug fix for HintType::L2PayloadWitness proactive hints
- reth v1.9.1 -> v1.9.3, SP1-CC updated
- Experimental witness endpoint enabled
- Proxy updated to v2.6.0 with V4 certificate support

* refactor: replace kona-rpc with local RPC types

Remove kona-rpc dependency to avoid rollup-boost -> reth-optimism-primitives
dependency chain, which causes alloy version conflicts with hokulea v1.1.4.

The rollup-boost crate depends on reth v1.9.3 which expects alloy 1.0.x,
incompatible with alloy 1.4.x required by hokulea's sp1-contract-call deps.

Since op-succinct only uses OutputResponse and SafeHeadResponse from kona-rpc,
these simple types are now defined locally in utils/host/src/rpc_types.rs.

* chore: bump elfs

* feat(fault-proof): add vkey, rollup config hash validation to prevent proving incompatible games (#758)

* feat(fault-proof): add vkey validation to prevent proving incompatible games

* feat(fault-proof): add prove-only mode

* test: add missing field

* test: add prove only mode behavior test

* feat: self-adapting impl

* test(e2e): add vkey mismatch game creation skip test

* chore: trigger CI (submodule now available)

* fix(contracts): resolve stack-too-deep in DeployGameImplOnly

Inline vm.env*() calls directly into constructor to reduce local
variables from 12+ to 2, staying within EVM's 16-slot stack limit.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix(tests): resolve compilation errors

* test(fault-proof): verify old proposer does not defend new games

* chore: fmt

* test: fix use real vkeys for integration tests

* feat: add rollup config hash to proposer id

* fix(fault-proof): address review comments for vkey validation PR

- Simplify ProposerIdentity struct by removing redundant fields
- Add rollup_config_hash to on_chain_vkeys_match for consistency with is_owned
- Consolidate ownership check into should_skip_proving method
- Remove DeployGameImplOnly.s.sol, use UpgradeOPSuccinctFDG.s.sol instead
- Update env.rs and justfile to use upgrade script
- Bump BACKUP_VERSION to 2 for new Game fields
- Remove unnecessary comment in constants.rs

* refactor(tests): use Forge script for game impl upgrade in E2E tests

Addresses review comment to use the same Forge script that operators
use in production (UpgradeOPSuccinctFDG.s.sol) instead of Go-based
deployment logic.

Changes:
- Add UpgradeGameImplConfig and UpgradeGameImpl to sysgo (succinctlabs/optimism#336)
- Replace DeployGameImplWithFakeVkeys + UpgradeGameImplementation with
  single UpgradeGameImplWithFakeVkeys that calls the Forge script
- Update E2E test to use the new combined method

* fix(ci): formatting and integration test block offset

- Run cargo fmt to fix line formatting in proposer.rs and env.rs
- Fix integration test block offset from +1 to +10 to match other tests

* fix(contracts): return gameImpl address from UpgradeOPSuccinctFDG script

The Rust integration test expects the Forge script to return the
gameImpl address in JSON output. Add return value to run() function.

* fix(tests): increase future block offset to avoid race condition

The test_proposer_rejects_future_starting_block test was flaky on
eigenda because the L2 chain could advance during test setup. With the
PR's added identity computation overhead, there was enough time for
the eigenda chain to catch up and invalidate the "future block" test
scenario.

Increased the offset from 100 to 10000 blocks to provide ~5.5 hours of
buffer with 2-second blocks, ensuring the test is robust regardless of
setup time or block production rate.

* fix(fault-proof): address review comments for vkey validation PR

- Revert BACKUP_VERSION to 1 (no release since backup feature added)
- Replace unwrap() with proper error handling in aggregation vkey decode
- Fix agg_vk.vk.bytes32() to agg_vk.bytes32() for consistency

* docs(fault-proof): add hardfork transition workflow documentation

Document the vkey validation feature that enables zero-downtime hardfork
transitions. Covers proposer identity, owned vs foreign games, hardfork
detection, and step-by-step transition workflow for operators.

* chore: re-trigger CI after pushing submodule commit

* fix(tests): fix compilation errors after rebase

- Remove unused import start_proposer_prove_only from env.rs
- Fix setImplementationCall to setImplementation_0Call (op-contracts v5.0.0)
- Add prove_only_mode field to ProposerConfig in process.rs

* refactor(fault-proof): remove unused prove_only_mode, improve logging

- Remove prove_only_mode config field (vkey validation handles this automatically)
- Promote foreign game skip log from DEBUG to INFO for better visibility during hardfork transitions

* refactor(proposer): simplify vkey computation using bytes32_raw/hash_bytes

Use SP1 SDK's direct byte conversion methods instead of hex roundtrip.

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* chore: bump tests/optimism to tip of op-succinct-sysgo (#785)

Updates the submodule to point to the properly merged commit on
op-succinct-sysgo branch, replacing the orphaned merge commit that
was created before PR #336 was merged.

* feat: add backport automation for multi-version release management (#789)

- Add GitHub Action workflow using tibdex/backport@v2
- Detects backport/* labels on merged PRs to main
- Automatically cherry-picks changes to maintenance branches
- Posts comments on original PR if conflicts detected
- Update CONTRIBUTING.md with backporting guidelines

* test(precompiles): add cycle tracker regression tests (#772)

* test(precompiles): add cycle tracker regression tests

Add unit tests for precompile cycle tracking to prevent regressions
like the one in commit d3b31279 where cycle tracking was dropped.

Changes:
- Add shared `cycle_tracker` constants module with names and keys
- Refactor `get_precompile_tracker_name…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mw2000 mw2000 Awaiting requested review from mw2000

Assignees

No one assigned

Labels

stale Flag: Stale issue or PR

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

feat(flashblocks): Streaming State Root Calculation

4 participants

@Farukest @cb-heimdall @mw2000 @refcell