Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Electra alpha8 spec updates #6496

Open
wants to merge 30 commits into
base: unstable
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
3c24e2b
Fix partial withdrawals count
pawanjay176 Oct 10, 2024
3fa7790
Remove get_active_balance
pawanjay176 Oct 10, 2024
3d595e6
Remove queue_entire_balance_and_reset_validator
pawanjay176 Oct 10, 2024
0a4e6be
Switch to compounding when consolidating with source==target
pawanjay176 Oct 10, 2024
c0cac0e
Queue deposit requests and apply them during epoch processing
pawanjay176 Oct 12, 2024
b28d010
Fix ef tests
pawanjay176 Oct 16, 2024
c8dfe7d
Clear todos
pawanjay176 Oct 16, 2024
0dd215c
Fix engine api formatting issues
pawanjay176 Oct 17, 2024
061be34
Merge branch 'unstable' into electra-alpha7
pawanjay176 Oct 29, 2024
37c7658
Make add_validator_to_registry more in line with the spec
pawanjay176 Oct 29, 2024
9f85074
Address some review comments
pawanjay176 Oct 29, 2024
707338d
Cleanup
pawanjay176 Oct 29, 2024
b2b728d
Update initialize_beacon_state_from_eth1
pawanjay176 Oct 29, 2024
21da047
Merge branch 'unstable' into electra-alpha7
pawanjay176 Nov 5, 2024
2aaf9b5
Fix rpc decoding for blobs by range/root
pawanjay176 Nov 6, 2024
27ce1a0
Fix block hash computation
pawanjay176 Nov 7, 2024
da953b8
Fix process_deposits bug
pawanjay176 Nov 8, 2024
6c56dc4
Merge branch 'unstable' into electra-alpha7
pawanjay176 Nov 20, 2024
def2498
Fix topup deposit processing bug
pawanjay176 Nov 26, 2024
c5c9aca
Update builder api for electra
pawanjay176 Dec 4, 2024
6d10456
Refactor mock builder to separate functionality
pawanjay176 Dec 5, 2024
e49689b
Merge branch 'unstable' into electra-alpha7
pawanjay176 Dec 11, 2024
140d4cf
Address review comments
pawanjay176 Dec 12, 2024
da33bd6
Use copied for reference rather than cloned
michaelsproul Dec 13, 2024
0af6b0f
Optimise and simplify PendingDepositsContext::new
michaelsproul Dec 13, 2024
1af232b
Merge remote-tracking branch 'origin/unstable' into electra-alpha7
michaelsproul Dec 13, 2024
21591a3
Fix processing of deposits with invalid signatures
michaelsproul Dec 13, 2024
d5ec058
Remove redundant code in genesis init
pawanjay176 Dec 14, 2024
5a3f5df
Revert "Refactor mock builder to separate functionality"
pawanjay176 Dec 14, 2024
ff687ed
Revert "Update builder api for electra"
pawanjay176 Dec 14, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 26 additions & 18 deletions beacon_node/execution_layer/src/block_hash.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ use keccak_hash::KECCAK_EMPTY_LIST_RLP;
use triehash::ordered_trie_root;
use types::{
EncodableExecutionBlockHeader, EthSpec, ExecutionBlockHash, ExecutionBlockHeader,
ExecutionPayloadRef, Hash256,
ExecutionPayloadRef, ExecutionRequests, Hash256,
};

/// Calculate the block hash of an execution block.
Expand All @@ -17,6 +17,7 @@ use types::{
pub fn calculate_execution_block_hash<E: EthSpec>(
payload: ExecutionPayloadRef<E>,
parent_beacon_block_root: Option<Hash256>,
execution_requests: Option<&ExecutionRequests<E>>,
) -> (ExecutionBlockHash, Hash256) {
// Calculate the transactions root.
// We're currently using a deprecated Parity library for this. We should move to a
Expand All @@ -38,6 +39,7 @@ pub fn calculate_execution_block_hash<E: EthSpec>(

let rlp_blob_gas_used = payload.blob_gas_used().ok();
let rlp_excess_blob_gas = payload.excess_blob_gas().ok();
let requests_root = execution_requests.map(|requests| requests.requests_hash());

// Construct the block header.
let exec_block_header = ExecutionBlockHeader::from_payload(
Expand All @@ -48,6 +50,7 @@ pub fn calculate_execution_block_hash<E: EthSpec>(
rlp_blob_gas_used,
rlp_excess_blob_gas,
parent_beacon_block_root,
requests_root,
);

// Hash the RLP encoding of the block header.
Expand Down Expand Up @@ -118,6 +121,7 @@ mod test {
blob_gas_used: None,
excess_blob_gas: None,
parent_beacon_block_root: None,
requests_root: None,
};
let expected_rlp = "f90200a0e0a94a7a3c9617401586b1a27025d2d9671332d22d540e0af72b069170380f2aa01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d4934794ba5e000000000000000000000000000000000000a0ec3c94b18b8a1cff7d60f8d258ec723312932928626b4c9355eb4ab3568ec7f7a050f738580ed699f0469702c7ccc63ed2e51bc034be9479b7bff4e68dee84accfa029b0562f7140574dd0d50dee8a271b22e1a0a7b78fca58f7c60370d8317ba2a9b9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000830200000188016345785d8a00008301553482079e42a0000000000000000000000000000000000000000000000000000000000000000088000000000000000082036b";
let expected_hash =
Expand Down Expand Up @@ -149,6 +153,7 @@ mod test {
blob_gas_used: None,
excess_blob_gas: None,
parent_beacon_block_root: None,
requests_root: None,
};
let expected_rlp = "f901fda0927ca537f06c783a3a2635b8805eef1c8c2124f7444ad4a3389898dd832f2dbea01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d4934794ba5e000000000000000000000000000000000000a0e97859b065bd8dbbb4519c7cb935024de2484c2b7f881181b4360492f0b06b82a050f738580ed699f0469702c7ccc63ed2e51bc034be9479b7bff4e68dee84accfa029b0562f7140574dd0d50dee8a271b22e1a0a7b78fca58f7c60370d8317ba2a9b9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000800188016345785d8a00008301553482079e42a0000000000000000000000000000000000000000000000000000000000002000088000000000000000082036b";
let expected_hash =
Expand Down Expand Up @@ -181,6 +186,7 @@ mod test {
blob_gas_used: None,
excess_blob_gas: None,
parent_beacon_block_root: None,
requests_root: None,
};
let expected_hash =
Hash256::from_str("6da69709cd5a34079b6604d29cd78fc01dacd7c6268980057ad92a2bede87351")
Expand Down Expand Up @@ -211,6 +217,7 @@ mod test {
blob_gas_used: Some(0x0u64),
excess_blob_gas: Some(0x0u64),
parent_beacon_block_root: Some(Hash256::from_str("f7d327d2c04e4f12e9cdd492e53d39a1d390f8b1571e3b2a22ac6e1e170e5b1a").unwrap()),
requests_root: None,
};
let expected_hash =
Hash256::from_str("a7448e600ead0a23d16f96aa46e8dea9eef8a7c5669a5f0a5ff32709afe9c408")
Expand All @@ -221,29 +228,30 @@ mod test {
#[test]
fn test_rlp_encode_block_electra() {
let header = ExecutionBlockHeader {
parent_hash: Hash256::from_str("172864416698b842f4c92f7b476be294b4ef720202779df194cd225f531053ab").unwrap(),
parent_hash: Hash256::from_str("a628f146df398a339768bd101f7dc41d828be79aca5dd02cc878a51bdbadd761").unwrap(),
ommers_hash: Hash256::from_str("1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347").unwrap(),
beneficiary: Address::from_str("878705ba3f8bc32fcf7f4caa1a35e72af65cf766").unwrap(),
state_root: Hash256::from_str("c6457d0df85c84c62d1c68f68138b6e796e8a44fb44de221386fb2d5611c41e0").unwrap(),
transactions_root: Hash256::from_str("56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421").unwrap(),
receipts_root: Hash256::from_str("56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421").unwrap(),
logs_bloom:<[u8; 256]>::from_hex("00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000").unwrap().into(),
beneficiary: Address::from_str("f97e180c050e5ab072211ad2c213eb5aee4df134").unwrap(),
state_root: Hash256::from_str("fdff009f8280bd113ebb4df8ce4e2dcc9322d43184a0b506e70b7f4823ca1253").unwrap(),
transactions_root: Hash256::from_str("452806578b4fa881cafb019c47e767e37e2249accf859159f00cddefb2579bb5").unwrap(),
receipts_root: Hash256::from_str("72ceac0f16a32041c881b3220d39ca506a286bef163c01a4d0821cd4027d31c7").unwrap(),
logs_bloom:<[u8; 256]>::from_hex("10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000400000000000000000000000020000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000").unwrap().into(),
difficulty: Uint256::ZERO,
number: Uint256::from(97),
gas_limit: Uint256::from(27482534),
gas_used: Uint256::ZERO,
timestamp: 1692132829u64,
extra_data: hex::decode("d883010d00846765746888676f312e32302e37856c696e7578").unwrap(),
mix_hash: Hash256::from_str("0b493c22d2ad4ca76c77ae6ad916af429b42b1dc98fdcb8e5ddbd049bbc5d623").unwrap(),
number: Uint256::from(8230),
gas_limit: Uint256::from(30000000),
gas_used: Uint256::from(3716848),
timestamp: 1730921268,
extra_data: hex::decode("d883010e0c846765746888676f312e32332e32856c696e7578").unwrap(),
mix_hash: Hash256::from_str("e87ca9a45b2e61bbe9080d897db1d584b5d2367d22e898af901091883b7b96ec").unwrap(),
nonce: Hash64::ZERO,
base_fee_per_gas: Uint256::from(2374u64),
base_fee_per_gas: Uint256::from(7u64),
withdrawals_root: Some(Hash256::from_str("56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421").unwrap()),
blob_gas_used: Some(0x0u64),
excess_blob_gas: Some(0x0u64),
parent_beacon_block_root: Some(Hash256::from_str("f7d327d2c04e4f12e9cdd492e53d39a1d390f8b1571e3b2a22ac6e1e170e5b1a").unwrap()),
blob_gas_used: Some(786432),
excess_blob_gas: Some(44695552),
parent_beacon_block_root: Some(Hash256::from_str("f3a888fee010ebb1ae083547004e96c254b240437823326fdff8354b1fc25629").unwrap()),
requests_root: Some(Hash256::from_str("9440d3365f07573919e1e9ac5178c20ec6fe267357ee4baf8b6409901f331b62").unwrap()),
};
let expected_hash =
Hash256::from_str("a7448e600ead0a23d16f96aa46e8dea9eef8a7c5669a5f0a5ff32709afe9c408")
Hash256::from_str("61e67afc96bf21be6aab52c1ace1db48de7b83f03119b0644deb4b69e87e09e1")
michaelsproul marked this conversation as resolved.
Show resolved Hide resolved
.unwrap();
test_rlp_encoding(&header, None, expected_hash);
}
Expand Down
2 changes: 1 addition & 1 deletion beacon_node/execution_layer/src/engine_api/http.rs
Original file line number Diff line number Diff line change
Expand Up @@ -812,7 +812,7 @@ impl HttpJsonRpc {
new_payload_request_electra.versioned_hashes,
new_payload_request_electra.parent_beacon_block_root,
new_payload_request_electra
.execution_requests_list
.execution_requests
.get_execution_requests_list(),
]);

Expand Down
30 changes: 7 additions & 23 deletions beacon_node/execution_layer/src/engine_api/json_structures.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@ use strum::EnumString;
use superstruct::superstruct;
use types::beacon_block_body::KzgCommitments;
use types::blob_sidecar::BlobsList;
use types::execution_requests::{ConsolidationRequests, DepositRequests, WithdrawalRequests};
use types::execution_requests::{
ConsolidationRequests, DepositRequests, RequestPrefix, WithdrawalRequests,
};
use types::{Blob, FixedVector, KzgProof, Unsigned};

#[derive(Debug, PartialEq, Serialize, Deserialize)]
Expand Down Expand Up @@ -339,25 +341,6 @@ impl<E: EthSpec> From<JsonExecutionPayload<E>> for ExecutionPayload<E> {
}
}

/// This is used to index into the `execution_requests` array.
#[derive(Debug, Copy, Clone)]
enum RequestPrefix {
Deposit,
Withdrawal,
Consolidation,
}

impl RequestPrefix {
pub fn from_prefix(prefix: u8) -> Option<Self> {
match prefix {
0 => Some(Self::Deposit),
1 => Some(Self::Withdrawal),
2 => Some(Self::Consolidation),
_ => None,
}
}
}

/// Format of `ExecutionRequests` received over the engine api.
///
/// Array of ssz-encoded requests list encoded as hex bytes.
Expand All @@ -379,7 +362,8 @@ impl<E: EthSpec> TryFrom<JsonExecutionRequests> for ExecutionRequests<E> {

for (i, request) in value.0.into_iter().enumerate() {
// hex string
let decoded_bytes = hex::decode(request).map_err(|e| format!("Invalid hex {:?}", e))?;
let decoded_bytes = hex::decode(request.strip_prefix("0x").unwrap_or(&request))
.map_err(|e| format!("Invalid hex {:?}", e))?;
match RequestPrefix::from_prefix(i as u8) {
Some(RequestPrefix::Deposit) => {
requests.deposits = DepositRequests::<E>::from_ssz_bytes(&decoded_bytes)
Expand Down Expand Up @@ -431,7 +415,7 @@ pub struct JsonGetPayloadResponse<E: EthSpec> {
#[superstruct(only(V3, V4))]
pub should_override_builder: bool,
#[superstruct(only(V4))]
pub requests: JsonExecutionRequests,
pub execution_requests: JsonExecutionRequests,
}

impl<E: EthSpec> TryFrom<JsonGetPayloadResponse<E>> for GetPayloadResponse<E> {
Expand Down Expand Up @@ -464,7 +448,7 @@ impl<E: EthSpec> TryFrom<JsonGetPayloadResponse<E>> for GetPayloadResponse<E> {
block_value: response.block_value,
blobs_bundle: response.blobs_bundle.into(),
should_override_builder: response.should_override_builder,
requests: response.requests.try_into()?,
requests: response.execution_requests.try_into()?,
}))
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ pub struct NewPayloadRequest<'block, E: EthSpec> {
#[superstruct(only(Deneb, Electra))]
pub parent_beacon_block_root: Hash256,
#[superstruct(only(Electra))]
pub execution_requests_list: &'block ExecutionRequests<E>,
pub execution_requests: &'block ExecutionRequests<E>,
}

impl<'block, E: EthSpec> NewPayloadRequest<'block, E> {
Expand Down Expand Up @@ -121,8 +121,11 @@ impl<'block, E: EthSpec> NewPayloadRequest<'block, E> {

let _timer = metrics::start_timer(&metrics::EXECUTION_LAYER_VERIFY_BLOCK_HASH);

let (header_hash, rlp_transactions_root) =
calculate_execution_block_hash(payload, parent_beacon_block_root);
let (header_hash, rlp_transactions_root) = calculate_execution_block_hash(
payload,
parent_beacon_block_root,
self.execution_requests().ok().cloned(),
);

if header_hash != self.block_hash() {
return Err(Error::BlockHashMismatch {
Expand Down Expand Up @@ -185,7 +188,7 @@ impl<'a, E: EthSpec> TryFrom<BeaconBlockRef<'a, E>> for NewPayloadRequest<'a, E>
.map(kzg_commitment_to_versioned_hash)
.collect(),
parent_beacon_block_root: block_ref.parent_root,
execution_requests_list: &block_ref.body.execution_requests,
execution_requests: &block_ref.body.execution_requests,
})),
}
}
Expand Down
2 changes: 1 addition & 1 deletion beacon_node/execution_layer/src/test_utils/handle_rpc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -374,7 +374,7 @@ pub async fn handle_rpc<E: EthSpec>(
.into(),
should_override_builder: false,
// TODO(electra): add EL requests in mock el
requests: Default::default(),
execution_requests: Default::default(),
})
.unwrap()
}
Expand Down
4 changes: 2 additions & 2 deletions beacon_node/store/src/partial_beacon_state.rs
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ where
pub earliest_consolidation_epoch: Epoch,

#[superstruct(only(Electra))]
pub pending_balance_deposits: List<PendingBalanceDeposit, E::PendingBalanceDepositsLimit>,
pub pending_deposits: List<PendingDeposit, E::PendingDepositsLimit>,
#[superstruct(only(Electra))]
pub pending_partial_withdrawals:
List<PendingPartialWithdrawal, E::PendingPartialWithdrawalsLimit>,
Expand Down Expand Up @@ -403,7 +403,7 @@ impl<E: EthSpec> TryInto<BeaconState<E>> for PartialBeaconState<E> {
earliest_exit_epoch,
consolidation_balance_to_consume,
earliest_consolidation_epoch,
pending_balance_deposits,
pending_deposits,
pending_partial_withdrawals,
pending_consolidations
],
Expand Down
13 changes: 13 additions & 0 deletions consensus/state_processing/src/genesis.rs
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,19 @@ pub fn initialize_beacon_state_from_eth1<E: EthSpec>(
// Remove intermediate Deneb fork from `state.fork`.
state.fork_mut().previous_version = spec.electra_fork_version;

// iterate over an empty pending_deposits list here and increase balance
let pending_deposits = state.pending_deposits()?.clone();
for pending_deposit in pending_deposits.into_iter() {
if let Some(validator_index) = state.pubkey_cache().get(&pending_deposit.pubkey) {
state
.balances_mut()
.get_mut(validator_index)
.ok_or(Error::UnknownValidator(validator_index))?
.safe_add(pending_deposit.amount)?;
}
}
*state.pending_deposits_mut()? = List::empty();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this doing? How can we be iterating over an empty list -- doesn't this just not do anything?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding was this applies pending_deposits to the state during initialization if the state contains pending_deposits. Here's the part of the spec

for deposit in state.pending_deposits:
        validator_index = ValidatorIndex(validator_pubkeys.index(deposit.pubkey))
        increase_balance(state, validator_index, deposit.amount)
    state.pending_deposits = []

Copy link
Member

@michaelsproul michaelsproul Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh right, I was confused because we are calling upgrade_state_to_electra immediately prior to this, which sets the pending_deposits to the empty list:

let post = upgrade_state_to_electra(&mut state, Epoch::new(0), Epoch::new(0), spec)?;

pending_deposits: Default::default(),

I think we might need to move the apply_deposit logic after upgrading to Electra, so that the pending_deposits list gets populated. Currently it happens here:

for deposit in deposits.into_iter() {
deposit_tree
.push_leaf(deposit.data.tree_hash_root())
.map_err(BlockProcessingError::MerkleTreeError)?;
state.eth1_data_mut().deposit_root = deposit_tree.root();
let Deposit { proof, data } = deposit;
apply_deposit(&mut state, data, Some(proof), true, spec)?;
}

Or maybe we just delete the whole initialize_beacon_state_from_eth1 function and say we don't support starting from eth1 genesis any more 😅

It's interesting that the EF tests for initialize_beacon_state_from_eth1 already pass. Maybe processing the deposits with the pre-Electra logic is actually equivalent, and we could just delete the code here that loops over pending_deposits?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah i'm okay with that. It seemed redundant to me initially but i just implemented it for completeness.
I'm fine with removing it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in d5ec058


// TODO(electra): think about this more and determine the best way to
// do this. The spec tests will expect that the sync committees are
// calculated using the electra value for MAX_EFFECTIVE_BALANCE when
Expand Down
4 changes: 3 additions & 1 deletion consensus/state_processing/src/per_block_processing.rs
Original file line number Diff line number Diff line change
Expand Up @@ -514,6 +514,7 @@ pub fn get_expected_withdrawals<E: EthSpec>(
// Consume pending partial withdrawals
let partial_withdrawals_count =
if let Ok(partial_withdrawals) = state.pending_partial_withdrawals() {
let mut partial_withdrawals_count = 0;
for withdrawal in partial_withdrawals {
if withdrawal.withdrawable_epoch > epoch
|| withdrawals.len() == spec.max_pending_partials_per_withdrawals_sweep as usize
Expand Down Expand Up @@ -546,8 +547,9 @@ pub fn get_expected_withdrawals<E: EthSpec>(
});
withdrawal_index.safe_add_assign(1)?;
}
partial_withdrawals_count.safe_add_assign(1)?;
}
Some(withdrawals.len())
Some(partial_withdrawals_count)
} else {
None
};
Expand Down
Loading
Loading