Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Encapsulate field parsing #4

Merged
merged 3 commits into from
Sep 29, 2023
Merged

Encapsulate field parsing #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a80dcd7
Encapsulate field parsing
tuommaki Sep 28, 2023
eb22dd8
Expose `decompress_bytecode` to outside
tuommaki Sep 29, 2023
e496e6b
Drop unnecessary `.into()`
tuommaki Sep 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
261 changes: 13 additions & 248 deletions src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
#![feature(array_chunks)]

use std::collections::HashMap;
use std::fs;
use thiserror::Error;

mod state;
use crate::state::CommitBlockInfoV1;
Expand All @@ -19,21 +17,6 @@ pub const ZK_SYNC_ADDR: &str = "0x32400084C286CF3E17e7B677ea9583e60a000324";
pub const GENESIS_BLOCK: u64 = 16_627_460;
pub const BLOCK_STEP: u64 = 128;

#[derive(Error, Debug)]
pub enum ParseError {
#[error("invalid Calldata: {0}")]
InvalidCalldata(String),

#[error("invalid StoredBlockInfo: {0}")]
InvalidStoredBlockInfo(String),

#[error("invalid CommitBlockInfo: {0}")]
InvalidCommitBlockInfo(String),

#[error("invalid compressed bytecode: {0}")]
InvalidCompressedByteCode(String),
}

pub fn create_initial_state() {
let _input = fs::read_to_string(INITAL_STATE_PATH).unwrap();
todo!();
Expand All @@ -52,10 +35,10 @@ pub async fn init_eth_adapter(http_url: &str) -> (Provider<Http>, Contract) {
fn parse_calldata(commit_blocks_fn: &Function, calldata: &[u8]) -> Result<Vec<CommitBlockInfoV1>> {
let mut parsed_input = commit_blocks_fn
.decode_input(&calldata[4..])
.map_err(|e| ParseError::InvalidCalldata(e.to_string()))?;
.map_err(|e| state::ParseError::InvalidCalldata(e.to_string()))?;
Copy link
Collaborator Author

@tuommaki tuommaki Sep 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These shared error types here are ugly as hell and don't feel right, but let's fix these separately, if that's ok?


if parsed_input.len() != 2 {
return Err(ParseError::InvalidCalldata(format!(
return Err(state::ParseError::InvalidCalldata(format!(
"invalid number of parameters (got {}, expected 2) for commitBlocks function",
parsed_input.len()
))
Expand All @@ -64,24 +47,26 @@ fn parse_calldata(commit_blocks_fn: &Function, calldata: &[u8]) -> Result<Vec<Co

let new_blocks_data = parsed_input
.pop()
.ok_or_else(|| ParseError::InvalidCalldata("new blocks data".to_string()))?;
.ok_or_else(|| state::ParseError::InvalidCalldata("new blocks data".to_string()))?;
let stored_block_info = parsed_input
.pop()
.ok_or_else(|| ParseError::InvalidCalldata("stored block info".to_string()))?;
.ok_or_else(|| state::ParseError::InvalidCalldata("stored block info".to_string()))?;

let abi::Token::Tuple(stored_block_info) = stored_block_info else {
return Err(ParseError::InvalidCalldata("invalid StoredBlockInfo".to_string()).into());
return Err(
state::ParseError::InvalidCalldata("invalid StoredBlockInfo".to_string()).into(),
);
};

let abi::Token::Uint(previous_l2_block_number) = stored_block_info[0].clone() else {
return Err(ParseError::InvalidStoredBlockInfo(
return Err(state::ParseError::InvalidStoredBlockInfo(
"cannot parse previous L2 block number".to_string(),
)
.into());
};

let abi::Token::Uint(_previous_enumeration_index) = stored_block_info[2].clone() else {
return Err(ParseError::InvalidStoredBlockInfo(
return Err(state::ParseError::InvalidStoredBlockInfo(
"cannot parse previous enumeration index".to_string(),
)
.into());
Expand All @@ -98,242 +83,22 @@ fn parse_commit_block_info(data: &abi::Token) -> Result<Vec<CommitBlockInfoV1>>
let mut res = vec![];

let abi::Token::Array(data) = data else {
return Err(ParseError::InvalidCommitBlockInfo(
return Err(state::ParseError::InvalidCommitBlockInfo(
"cannot convert newBlocksData to array".to_string(),
)
.into());
};

for data in data.iter() {
let abi::Token::Tuple(block_elems) = data else {
return Err(ParseError::InvalidCommitBlockInfo("struct elements".to_string()).into());
};
let abi::Token::Uint(new_l2_block_number) = block_elems[0].clone() else {
return Err(ParseError::InvalidCommitBlockInfo("blockNumber".to_string()).into());
};

/* TODO(tuommaki): Fix the check below.
if new_l2_block_number <= latest_l2_block_number {
println!("skipping before we even get started");
continue;
match CommitBlockInfoV1::try_from(data) {
Ok(blk) => res.push(blk),
Err(e) => println!("failed to parse commit block info: {}", e),
}
Comment on lines +93 to 96
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
match CommitBlockInfoV1::try_from(data) {
Ok(blk) => res.push(blk),
Err(e) => println!("failed to parse commit block info: {}", e),
}
if let Ok(block) = CommitBlockInfoV1::try_from(data) {
res.push(block);
}

Don't think there's any need to print out the errors, I think we'd rather implement some tracing later down the line and log that error or even panic! since that function should only ever fail on malformed data.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather keep this. In my experience these can easily save hours of debugging during development. Often times, some very minor unrelated mistake happens and if there's no observability, one really has to debug weird behaviour until the root cause is found. Having this kind of debug print shortens time of diagnostics a lot. Referring to earlier error handling, I'd leave panic! completely out of this code 🙂

*/

let abi::Token::Uint(timestamp) = block_elems[1].clone() else {
return Err(ParseError::InvalidCommitBlockInfo("timestamp".to_string()).into());
};

let abi::Token::Uint(new_enumeration_index) = block_elems[2].clone() else {
return Err(ParseError::InvalidCommitBlockInfo(
"indexRepeatedStorageChanges".to_string(),
)
.into());
};
let new_enumeration_index = new_enumeration_index.0[0];

let abi::Token::FixedBytes(state_root) = block_elems[3].clone() else {
return Err(ParseError::InvalidCommitBlockInfo("newStateRoot".to_string()).into());
};

let abi::Token::Uint(number_of_l1_txs) = block_elems[4].clone() else {
return Err(ParseError::InvalidCommitBlockInfo("numberOfLayer1Txs".to_string()).into());
};

let abi::Token::FixedBytes(l2_logs_tree_root) = block_elems[5].clone() else {
return Err(ParseError::InvalidCommitBlockInfo("l2LogsTreeRoot".to_string()).into());
};

let abi::Token::FixedBytes(priority_operations_hash) = block_elems[6].clone() else {
return Err(
ParseError::InvalidCommitBlockInfo("priorityOperationsHash".to_string()).into(),
);
};

let abi::Token::Bytes(initial_changes_calldata) = block_elems[7].clone() else {
return Err(
ParseError::InvalidCommitBlockInfo("initialStorageChanges".to_string()).into(),
);
};

if initial_changes_calldata.len() % 64 != 4 {
return Err(
ParseError::InvalidCommitBlockInfo("initialStorageChanges".to_string()).into(),
);
}
let abi::Token::Bytes(repeated_changes_calldata) = block_elems[8].clone() else {
return Err(
ParseError::InvalidCommitBlockInfo("repeatedStorageChanges".to_string()).into(),
);
};

let abi::Token::Bytes(l2_logs) = block_elems[9].clone() else {
return Err(ParseError::InvalidCommitBlockInfo("l2Logs".to_string()).into());
};

// TODO(tuommaki): Are these useful at all?
/*
let abi::Token::Bytes(_l2_arbitrary_length_msgs) = block_elems[10].clone() else {
return Err(ParseError::InvalidCommitBlockInfo(
"l2ArbitraryLengthMessages".to_string(),
)
.into());
};
*/

// TODO(tuommaki): Parse factory deps
let abi::Token::Array(factory_deps) = block_elems[11].clone() else {
return Err(ParseError::InvalidCommitBlockInfo("factoryDeps".to_string()).into());
};

let mut smartcontracts = vec![];
for bytecode in factory_deps.into_iter() {
let abi::Token::Bytes(bytecode) = bytecode else {
return Err(ParseError::InvalidCommitBlockInfo("factoryDeps".to_string()).into());
};

match decompress_bytecode(bytecode) {
Ok(bytecode) => smartcontracts.push(bytecode),
Err(e) => println!("failed to decompress bytecode: {}", e),
};
}

assert_eq!(repeated_changes_calldata.len() % 40, 4);

println!(
"Have {} new keys",
(initial_changes_calldata.len() - 4) / 64
);
println!(
"Have {} repeated keys",
(repeated_changes_calldata.len() - 4) / 40
);

let mut blk = CommitBlockInfoV1 {
block_number: new_l2_block_number.as_u64(),
timestamp: timestamp.as_u64(),
index_repeated_storage_changes: new_enumeration_index,
new_state_root: state_root,
number_of_l1_txs,
l2_logs_tree_root,
priority_operations_hash,
initial_storage_changes: HashMap::default(),
repeated_storage_changes: HashMap::default(),
l2_logs: l2_logs.to_vec(),
factory_deps: smartcontracts,
};

for initial_calldata in initial_changes_calldata[4..].chunks(64) {
let mut t = initial_calldata.array_chunks::<32>();
let key = *t.next().ok_or_else(|| {
ParseError::InvalidCommitBlockInfo("initialStorageChanges".to_string())
})?;
let value = *t.next().ok_or_else(|| {
ParseError::InvalidCommitBlockInfo("initialStorageChanges".to_string())
})?;

if t.next().is_some() {
return Err(ParseError::InvalidCommitBlockInfo(
"initialStorageChanges".to_string(),
)
.into());
}

let _ = blk.initial_storage_changes.insert(key, value);
}

for repeated_calldata in repeated_changes_calldata[4..].chunks(40) {
let index = u64::from_be_bytes([
repeated_calldata[0],
repeated_calldata[1],
repeated_calldata[2],
repeated_calldata[3],
repeated_calldata[4],
repeated_calldata[5],
repeated_calldata[6],
repeated_calldata[7],
]);
let mut t = repeated_calldata[8..].array_chunks::<32>();
let value = *t.next().ok_or_else(|| {
ParseError::InvalidCommitBlockInfo("repeatedStorageChanges".to_string())
})?;

if t.next().is_some() {
return Err(ParseError::InvalidCommitBlockInfo(
"repeatedStorageChanges".to_string(),
)
.into());
}

blk.repeated_storage_changes.insert(index, value);
}

// --------------------------------------------------------------
// TODO(tuommaki): Handle rest of the fields:
// - factoryDeps -> compressed bytecode
// --------------------------------------------------------------

res.push(blk);
}

Ok(res)
}

fn decompress_bytecode(data: Vec<u8>) -> Result<Vec<u8>> {
/*
let num_entries = u32::from_be_bytes([data[3], data[2], data[1], data[0]]);

*/
let mut offset = 0;

let dict_len = u16::from_be_bytes([data[offset + 1], data[offset]]);

offset += 2;

let end = 2 + dict_len as usize;
let dict = data[offset..end].to_vec();
offset += end;
let encoded_data = data[offset..].to_vec();

// Each dictionary element should be 8 bytes. Verify alignment.
if dict.len() % 8 != 0 {
return Err(ParseError::InvalidCompressedByteCode(format!(
"invalid dict length: {}",
dict.len()
))
.into());
}

let dict: Vec<&[u8]> = dict.chunks(8).collect();

// Verify that dictionary size is below maximum.
if dict.len() > (1 << 16)
/* 2^16 */
{
return Err(ParseError::InvalidCompressedByteCode(format!(
"too many elements in dictionary: {}",
dict.len()
))
.into());
}

let mut bytecode = vec![];
for idx in encoded_data.chunks(2) {
let idx = u16::from_be_bytes([idx[0], idx[1]]) as usize;
if dict.len() <= idx {
return Err(ParseError::InvalidCompressedByteCode(format!(
"encoded data index ({}) exceeds dictionary size ({})",
idx,
dict.len()
))
.into());
}

bytecode.append(&mut dict[idx].to_vec());
}

Ok(bytecode)
}

#[cfg(test)]
mod tests {
use ethers::{
Expand Down
Loading