Skip to content

Commit c0d8ad8

Browse files
prestwichclaude
prestwich
and
claude
committed
refactor: rewrite block processor and node to use signet-storage (#79)
* refactor: rewrite block processor to use signet-hot storage Replace the reth ProviderFactory-based storage with HotKv for rollup state reads. The processor becomes a stateless executor that reads from hot storage, runs the EVM, and returns ExecutedBlock. Extraction is moved to the node (PR3) to avoid lifetime issues with borrowed Extracts. - Replace Db: NodeTypesDbTrait generic with H: HotKv - Replace state_provider_database() with revm_state() using RevmRead - Remove on_host_commit() and commit_evm_results() - Add process_block() returning ExecutedBlock - Add build_executed_block() for type conversion - Remove signet-db, signet-node-types, reth-exex, reth-node-api deps - Add signet-hot, signet-storage-types deps - Remove Chain/PrimitivesOf/ExExNotification type aliases from lib.rs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: rewrite signet-node to use unified storage and rpc-storage Replace reth's ProviderFactory/BlockchainProvider with UnifiedStorage and swap signet-rpc for signet-rpc-storage. The node now holds Arc<UnifiedStorage<H>> and shares state with the RPC context through BlockTags (atomic block tag tracking) and broadcast::Sender (new block notifications). Key changes: - StorageRpcCtx accepts Arc<UnifiedStorage<H>> for shared ownership - Node struct uses HotKv generic instead of NodeTypesDbTrait - Block processing uses signet-extract's Extractor + ExtractableChainShim - Genesis loading via HistoryWrite::load_genesis + UnsafeDbWrite::commit - Fix metrics bug: record_notification_received now correctly increments the received counter instead of the processed counter Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: rewrite node-tests to use unified storage and align gas oracle Replace TmpDb/ProviderFactory with MemKv/UnifiedStorage in all node test infrastructure. Update signet-storage crates to v0.6.2 to fix MemKv intra-transaction read visibility. Align the cold-storage gas oracle with reth's GasPriceOracle by adding default_gas_price (1 Gwei), ignore_price (2 wei), and max_price (500 Gwei) to StorageRpcConfig. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: delete signet-db, signet-node-types, and signet-rpc These crates are fully replaced by signet-storage, signet-hot, and signet-rpc-storage respectively. Remove 6,700+ lines of dead code and clean up 22 unused workspace dependencies. Replace reth-db tempdir_path with tempfile in signet-node-config. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: rename signet-rpc-storage to signet-rpc Reclaim the signet-rpc name now that the old reth-backed crate is deleted. Rename directory, package, and all import paths. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: restore RUN EVM ascii art banner Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: resolve clippy empty_line_after_doc_comments warning in ascii banner Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address PR review — bugs, style, and test infra - Fix log_index per-receipt → per-block (kind.rs) - Fix hash/height mismatch in update_highest_processed_height (node.rs) - Fix serve_ipc panic: expect → ? (serve.rs) - Use seal_unchecked to avoid redundant header re-hashing (processor.rs) - Refactor imperative event loop → functional chain (processor.rs) - Add #[instrument(skip_all)] to run_evm and build_executed_block - Add TODO for two-reader consistency risk (processor.rs) - Group signet_hot imports across 6 RPC files - Move supported methods above "Unsupported" comment (eth/mod.rs) - Hoist function-scoped imports to module level (eth_rpc.rs) - Fix stale signet_rpc_storage reference (eth_rpc.rs) - Differentiate HTTP/WS doc comments, fix Result type (serve.rs) - Wrap cold storage polling loop in 30s timeout (context.rs) - Remove dead signet_events_in_range method (context.rs) - Update README to reflect new API types Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 47938d6 commit c0d8ad8

94 files changed

Lines changed: 2701 additions & 9853 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Cargo.toml‎

Lines changed: 4 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -36,14 +36,11 @@ incremental = false
3636
[workspace.dependencies]
3737
signet-blobber = { version = "0.16.0-rc.7", path = "crates/blobber" }
3838
signet-block-processor = { version = "0.16.0-rc.7", path = "crates/block-processor" }
39-
signet-db = { version = "0.16.0-rc.7", path = "crates/db" }
4039
signet-genesis = { version = "0.16.0-rc.7", path = "crates/genesis" }
4140
signet-node = { version = "0.16.0-rc.7", path = "crates/node" }
4241
signet-node-config = { version = "0.16.0-rc.7", path = "crates/node-config" }
4342
signet-node-tests = { version = "0.16.0-rc.7", path = "crates/node-tests" }
44-
signet-node-types = { version = "0.16.0-rc.7", path = "crates/node-types" }
4543
signet-rpc = { version = "0.16.0-rc.7", path = "crates/rpc" }
46-
signet-rpc-storage = { version = "0.16.0-rc.7", path = "crates/rpc-storage" }
4744

4845
init4-bin-base = { version = "0.18.0-rc.8", features = ["alloy"] }
4946

@@ -56,10 +53,10 @@ signet-tx-cache = "0.16.0-rc.8"
5653
signet-types = "0.16.0-rc.8"
5754
signet-zenith = "0.16.0-rc.8"
5855
signet-journal = "0.16.0-rc.8"
59-
signet-storage = "0.6.0"
60-
signet-cold = "0.6.0"
61-
signet-hot = "0.6.0"
62-
signet-storage-types = "0.6.0"
56+
signet-storage = "0.6.2"
57+
signet-cold = "0.6.2"
58+
signet-hot = "0.6.2"
59+
signet-storage-types = "0.6.2"
6360

6461
# ajj
6562
ajj = { version = "0.6.0" }
@@ -76,43 +73,22 @@ alloy = { version = "1.4.0", features = [
7673
"genesis",
7774
"arbitrary",
7875
] }
79-
alloy-contract = { version = "1.4.0", features = ["pubsub"] }
8076

8177
# Reth
8278
reth = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
83-
reth-ethereum = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
8479
reth-chainspec = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
85-
reth-codecs = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
86-
reth-db = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
87-
reth-db-api = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
88-
reth-db-common = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
89-
reth-eth-wire-types = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
90-
reth-evm-ethereum = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
9180
reth-exex = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
9281
reth-exex-test-utils = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
93-
reth-libmdbx = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
94-
reth-network-api = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
95-
reth-network-peers = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
9682
reth-node-api = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
97-
reth-node-ethereum = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
98-
reth-prune-types = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
99-
reth-rpc-eth-api = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
100-
reth-stages-types = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
10183
reth-transaction-pool = { git = "https://github.com/paradigmxyz/reth", tag = "v1.10.2" }
10284

10385
# Foundry periphery
10486
foundry-blob-explorers = "0.17"
10587

10688
# Async
10789
tokio = { version = "1.43.0", features = ["macros"] }
108-
async-trait = "0.1.87"
109-
110-
# Pinned for compatibility with reth
111-
parking_lot = "0.12"
11290

11391
# Misc
114-
chrono = "0.4.38"
115-
clap = "4"
11692
eyre = "0.6.12"
11793
futures-util = "0.3.31"
11894
hex = { package = "const-hex", version = "1.10", default-features = false, features = [
@@ -121,19 +97,15 @@ hex = { package = "const-hex", version = "1.10", default-features = false, featu
12197
itertools = "0.14.0"
12298
metrics = "0.24.2"
12399
openssl = { version = "0.10", features = ["vendored"] }
124-
proptest = "1.6.0"
125100
reqwest = "0.12.9"
126101
serde = { version = "1.0.217", features = ["derive"] }
127102
serde_json = "1.0.137"
128-
smallvec = "1.15.1"
129103
tracing = "0.1.41"
130104
tracing-subscriber = "0.3.19"
131105
thiserror = "2.0.12"
132106
url = "2.5.4"
133-
uuid = "1.16.0"
134107

135108
# Test Utils
136-
alloy-rlp = "0.3.11"
137109
tempfile = "3.17.0"
138110

139111
# [patch.crates-io]

‎crates/block-processor/Cargo.toml‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -20,14 +20,12 @@ init4-bin-base.workspace = true
2020
trevm.workspace = true
2121

2222
signet-blobber.workspace = true
23-
signet-db.workspace = true
24-
signet-node-types.workspace = true
23+
signet-hot.workspace = true
24+
signet-storage-types.workspace = true
2525

2626
alloy.workspace = true
2727

2828
reth.workspace = true
29-
reth-exex.workspace = true
30-
reth-node-api.workspace = true
3129
reth-chainspec.workspace = true
3230

3331
eyre.workspace = true

‎crates/block-processor/README.md‎

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,11 @@
11
# Signet Block Processor
22

3-
Block processing logic for the Signet Node. This crate takes a reth `Chain`,
4-
runs the Signet EVM, and commits the results to a database.
3+
Block processing logic for the Signet Node. This crate extracts and processes
4+
Signet blocks from host chain commits using the EVM, reading rollup state from
5+
hot storage.
56

67
# Significant Types
78

8-
- A few convenience type aliases:
9-
- `PrimitivesOf<Host>` - The primitives type used by the host.
10-
- `Chain<Host>` - A reth `Chain` using the host's primitives.
11-
- `ExExNotification<Host>` - A reth `ExExNotification` using the host's
12-
primitives.
13-
- `SignetBlockProcessorV1<Host, Db>` - The first version of the block processor.
9+
- `SignetBlockProcessor<H, Alias>` — The block processor. Reads state from
10+
`HotKv` storage, runs the EVM via `signet_evm`, and returns an
11+
`ExecutedBlock`.

‎crates/block-processor/src/lib.rs‎

Lines changed: 0 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,13 +21,3 @@ pub use utils::revm_spec;
2121

2222
mod v1;
2323
pub use v1::SignetBlockProcessor as SignetBlockProcessorV1;
24-
25-
/// Primitives used by the host.
26-
pub type PrimitivesOf<Host> =
27-
<<Host as reth_node_api::FullNodeTypes>::Types as reth_node_api::NodeTypes>::Primitives;
28-
29-
/// A [`reth::providers::Chain`] using the host primitives.
30-
pub type Chain<Host> = reth::providers::Chain<PrimitivesOf<Host>>;
31-
32-
/// A [`reth_exex::ExExNotification`] using the host primitives.
33-
pub type ExExNotification<Host> = reth_exex::ExExNotification<PrimitivesOf<Host>>;

‎crates/block-processor/src/metrics.rs‎

Lines changed: 0 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -57,9 +57,6 @@ const ENTER_TOKEN_PROCESSED_HELP: &str =
5757
const TRANSACT_PROCESSED: &str = "signet.block_processor.transact_events.processed";
5858
const TRANSACT_PROCESSED_HELP: &str = "Histogram of number of transact events processed per block";
5959

60-
const EXTRACTION_TIME: &str = "signet.block_processor.extraction.time";
61-
const EXTRACTION_TIME_HELP: &str = "Time taken to extract signet outputs from a host notification. Note: sometimes the extraction includes multiple blocks.";
62-
6360
const PROCESSING_TIME: &str = "signet.block_processor.processing.time";
6461
const PROCESSING_TIME_HELP: &str =
6562
"Time taken to process a single signet block from extracts, in milliseconds.";
@@ -80,7 +77,6 @@ static DESCRIBE: LazyLock<()> = LazyLock::new(|| {
8077
describe_histogram!(ENTER_PROCESSED, ENTER_PROCESSED_HELP);
8178
describe_histogram!(ENTER_TOKEN_PROCESSED, ENTER_TOKEN_PROCESSED_HELP);
8279
describe_histogram!(TRANSACT_PROCESSED, TRANSACT_PROCESSED_HELP);
83-
describe_histogram!(EXTRACTION_TIME, EXTRACTION_TIME_HELP);
8480
describe_histogram!(PROCESSING_TIME, PROCESSING_TIME_HELP);
8581
describe_histogram!(BLOCK_GAS_USED, BLOCK_GAS_USED_HELP);
8682
});
@@ -184,15 +180,6 @@ fn record_transacts_processed(value: u64) {
184180
transacts_processed().record(value as f64);
185181
}
186182

187-
fn extraction_time() -> Histogram {
188-
LazyLock::force(&DESCRIBE);
189-
histogram!(EXTRACTION_TIME)
190-
}
191-
192-
pub(crate) fn record_extraction_time(started_at: &std::time::Instant) {
193-
extraction_time().record(started_at.elapsed().as_millis() as f64);
194-
}
195-
196183
fn processing_time() -> Histogram {
197184
LazyLock::force(&DESCRIBE);
198185
histogram!(PROCESSING_TIME)

0 commit comments

Comments
 (0)