Skip to content

STR-1308: EE state diff prototype #800

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

evgenyzdanovich
Copy link
Contributor

Description

A prototype impl of EE DA using state diffs (without compression).

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature/Enhancement (non-breaking change which adds functionality or enhances an existing one)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor
  • New or updated tests
  • Dependency Update

Notes to Reviewers

Checklist

  • I have performed a self-review of my code.
  • I have commented my code where necessary.
  • I have updated the documentation if needed.
  • My changes do not introduce new warnings.
  • I have added (where necessary) tests that prove my changes are effective or that my feature works.
  • New and existing tests pass with my changes.

Related Issues

@evgenyzdanovich evgenyzdanovich requested review from a team as code owners May 5, 2025 20:43
@evgenyzdanovich evgenyzdanovich changed the title Str-1308: EE state diff prototype STR-1308: EE state diff prototype May 5, 2025
Copy link

codecov bot commented May 5, 2025

Codecov Report

Attention: Patch coverage is 80.56338% with 69 lines in your changes missing coverage. Please review.

Project coverage is 53.05%. Comparing base (eee4971) to head (d5e9a14).
Report is 10 commits behind head on main.

Files with missing lines Patch % Lines
crates/reth/exex/src/state_diff_exex.rs 0.00% 37 Missing ⚠️
bin/alpen-reth/src/main.rs 0.00% 19 Missing ⚠️
crates/reth/statediff/src/lib.rs 96.36% 8 Missing ⚠️
crates/reth/rpc/src/rpc.rs 0.00% 5 Missing ⚠️
@@            Coverage Diff             @@
##             main     #800      +/-   ##
==========================================
+ Coverage   52.72%   53.05%   +0.32%     
==========================================
  Files         300      302       +2     
  Lines       32940    33289     +349     
==========================================
+ Hits        17367    17660     +293     
- Misses      15573    15629      +56     
Files with missing lines Coverage Δ
crates/reth/db/src/rocksdb/db.rs 95.40% <100.00%> (+3.40%) ⬆️
crates/reth/rpc/src/rpc.rs 0.00% <0.00%> (ø)
crates/reth/statediff/src/lib.rs 96.36% <96.36%> (ø)
bin/alpen-reth/src/main.rs 0.00% <0.00%> (ø)
crates/reth/exex/src/state_diff_exex.rs 0.00% <0.00%> (ø)

... and 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

github-actions bot commented May 5, 2025

Commit: 6a432d0

SP1 Execution Results

program cycles success
Bitcoin Blockspace 71,567
EVM EE STF 277,604
CL STF 167,106
Checkpoint 4,252

Copy link
Contributor

@delbonis delbonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what from this design comes out of how the exex API is designed. Including tracking in the BatchStateDiff structure feels undesirable, see comments

I know this is a prototype and that's why we're using serde for it, but that seems like a huge overhead. At least adding #[serde(rename = ...)] to fields to make them encode as single chars would slim this down and skipping serializing entries in AccountInfo if they're unchanged.

Should we also include in the prototype a checker/applicator that applies the diff on top of a previous state and ensures it matches a final state? It would be nice to have that since that's how I imagine the flow in the proof would be more like, instead of recomputing the diff while checking the blocks and checking it matches a hash passed.

Comment on lines +28 to +45
/// N.B. "original" counterparts are needed to correctly construct the [`BatchStateDiff`]
/// for the range of blocks and not include changes for the keys whose values
/// were changed inside batch, but end up having the same value as before the batch.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like it's not just a diff then? We should have some builder type that remembers the original values and tracks modifications we're making, comparing against the originals, and updates an inner diff structure we extract at the end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, agree. Introduced a thin builder type that does pretty much the same.

Comment on lines +57 to +101
pub fn apply(&mut self, diff: BlockStateDiff) {
// Accumulate deployed contracts.
for (addr, bytecode) in diff.contracts.into_iter() {
self.contracts.insert(addr, bytecode);
}

for (addr, acc) in &diff.state {
// Update original account if seen for the first time in the batch.
if !self.original_accounts.contains_key(addr) {
self.original_accounts
.insert(*addr, acc.original_info.clone());
}

// Now, modify the actual account entry.
let cur_account = acc.account_info();
if &cur_account == self.original_accounts.get(addr).unwrap() {
// Remove if the actual account equals to the original.
self.accounts.remove(addr);
} else {
// The current account is different, update it.
self.accounts.insert(*addr, cur_account);
}

let original_acc_storage = self.original_storage_slots.entry(*addr).or_default();
let cur_acc_storage = self.storage_slots.entry(*addr).or_default();

for (key, value) in &acc.storage {
// Update original account storage if seen for the first time in the batch.
if !original_acc_storage.contains_key(key) {
original_acc_storage.insert(*key, value.original_value());
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's this weirdness here when we merge these diffs because we have to carry along the extra originals information inside the diff. What happens if the two diffs have different originals? That would probably be a bug, but this feels weird to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, this weirdness, IIUC, is imposed by the exex interface. If that happens - it pretty much means there's a bug in there.

@MdTeach MdTeach self-requested a review May 8, 2025 06:50
@evgenyzdanovich
Copy link
Contributor Author

@delbonis @MdTeach added some logic to reconstruct the state root based off of the batch diffs.
Please note - the logic is very raw and not yet tested, but feel free to take a look and leave suggestions.

I'm working on proper testing of reconstructing logic - both unit and esp fntests.

@evgenyzdanovich evgenyzdanovich force-pushed the str-1308-ee-state-diff-prototype branch from d5e9a14 to aef42c7 Compare May 14, 2025 16:52
@evgenyzdanovich evgenyzdanovich requested a review from a team as a code owner May 14, 2025 16:52
@evgenyzdanovich evgenyzdanovich requested a review from delbonis May 14, 2025 16:52
Copy link
Contributor

@delbonis delbonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Somewhat more minor changes.

I am curious about how this will lead into DA based sync? Do we have an alternate set of processing stages for that? I'm not sure what infra there is.

Comment on lines +18 to +19
/// block number => hash mapping for easier testing.
(BlockNumberByHash) u64 => Vec<u8>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be u64 => B256? And wouldn't it be called BlockHashByNumber? I'm not sure of the real schema here.

Comment on lines +17 to +21
#[derive(ThisError, Debug)]
/// A concrete error that may happen during state reconstruction.
pub enum StateError {
#[error("Mpt Error: {0}")]
MptError(#[from] strata_mpt::Error),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the main strata repo we've been doing this as

/// docs
#[derive(Debug, ThisError)]
pub enum StateError {
    #[error("mpt: {0}")]
    ...

not sure what the proper style in here ought be.


/// An (in-memory) representation of the EVM state reconstructed only from [`BatchStateDiff`].
#[derive(Clone, Default, Debug)]
pub struct StateReconstructed {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReconstructedState?

Also the order of items in this module is weird and split up into different parts.

Comment on lines +48 to +53
if account_info.is_none() {
// Account was actually destructed.
self.state_trie.delete(&acc_info_trie_path)?;
continue;
}
let account_info = account_info.unwrap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could write this as

let Some(account_info) = account_info else {
    // was deleted
    ...
    continue;
};

...

.map_err(to_jsonrpsee_error("Failed fetching block state diff"))
}

fn get_state_root_by_diffs(&self, block_number: u64) -> RpcResult<Option<B256>> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would call this _via_diffs. Saying "by" here implies that the diffs are a key we're looking up the state root by.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants