Electra alpha8 spec updates

[pawanjay176]

Issue Addressed

N/A

Proposed Changes

Implements changes for electra from https://github.com/ethereum/consensus-specs/releases/tag/v1.5.0-alpha.7 and https://github.com/ethereum/consensus-specs/releases/tag/v1.5.0-alpha.8

1. Fixed partial withdrawals count eip7251: Fix partial withdrawals count ethereum/consensus-specs#3943
2. Remove get_active_balance EIP-7251: Flatten get_active_balance ethereum/consensus-specs#3949
3. Remove queue_entire_balance_and_reset_validator Remove queue_entire_balance_and_reset_validator ethereum/consensus-specs#3951
4. Switch to compounding when consolidating with source==target eip7251: Switch to compounding when consolidating with source==target ethereum/consensus-specs#3918
5. Queue deposit requests and apply them during epoch processing eip6110: Queue deposit requests and apply them during epoch processing ethereum/consensus-specs#3818

Other changes from the releases are mostly cosmetic/ do not require changes on our end.
Can be reviewed per commit for each change. However, b28d010 contains bug fixes from all the changes

[pawanjay176]

Should we change the type to SignatureBytes for the other request types as well for consistency?

[ethDreamer]

curious why we did this?

[pawanjay176]

Because we need to allow for invalid bls signature bytes to be sent over the engine api. Otherwise, it becomes a deserialization error when we get the payload from the EL instead of an invalid deposit during processing

[pawanjay176]

I'm very sus about this. Seems to have fixed the deposits processing bug though
cc @michaelsproul

[pawanjay176]

For some added context, we want deposit processing to work when process_pending_deposits processes 2 deposits where deposit 1 is a new validator and deposit 2 is a topup for the new validator.
Since process_effective_balance_updates happens after process_pending_deposits, we need the effective balance update happening after adding the new validator

[michaelsproul]

I agree with this assessment of the situation. It's a bit gnarly that we need to do an effective balance update here outside the main single pass loop, but I think it's the best way.

I was thinking it might make sense to put this block of code after the logic for processing pending consolidations because that's the order used by the spec:

process_pending_deposits(state)  # [New in Electra:EIP7251]
process_pending_consolidations(state)  # [New in Electra:EIP7251]
process_effective_balance_updates(state)  # [Modified in Electra:EIP7251]

However, the spec also requires that consolidations are for known validators (both source and target), so that should not interact at all with our newly added validators.

[michaelsproul]

The only other thing I would consider doing would be getting rid of the added_validators vec and just running effective balance updates for all the validators from num_validators.. on wards.

Ideally we could even use something like iter_from that returns CoW pointers (iter_cow_from), however this doesn't exist in Milhouse yet.

[michaelsproul]

Opened an issue on Milhouse:

• Add iter_cow_from milhouse#57

[michaelsproul]

I've reviewed all but 2 files, but one of those files is single_pass 😅 Will come back for that tomorrow

[michaelsproul]

What is this doing? How can we be iterating over an empty list -- doesn't this just not do anything?

[pawanjay176]

My understanding was this applies pending_deposits to the state during initialization if the state contains pending_deposits. Here's the part of the spec

for deposit in state.pending_deposits:
        validator_index = ValidatorIndex(validator_pubkeys.index(deposit.pubkey))
        increase_balance(state, validator_index, deposit.amount)
    state.pending_deposits = []

[michaelsproul]

Ohh right, I was confused because we are calling upgrade_state_to_electra immediately prior to this, which sets the pending_deposits to the empty list:

lighthouse/consensus/state_processing/src/genesis.rs

Line 120 in 6d10456

let post = upgrade_state_to_electra(&mut state, Epoch::new(0), Epoch::new(0), spec)?;

lighthouse/consensus/state_processing/src/upgrade/electra.rs

Line 165 in 6d10456

pending_deposits: Default::default(),

I think we might need to move the apply_deposit logic after upgrading to Electra, so that the pending_deposits list gets populated. Currently it happens here:

lighthouse/consensus/state_processing/src/genesis.rs

Lines 36 to 43 in 6d10456

	for deposit in deposits.into_iter() {
	deposit_tree
	.push_leaf(deposit.data.tree_hash_root())
	.map_err(BlockProcessingError::MerkleTreeError)?;
	state.eth1_data_mut().deposit_root = deposit_tree.root();
	let Deposit { proof, data } = deposit;
	apply_deposit(&mut state, data, Some(proof), true, spec)?;
	}

Or maybe we just delete the whole initialize_beacon_state_from_eth1 function and say we don't support starting from eth1 genesis any more 😅

It's interesting that the EF tests for initialize_beacon_state_from_eth1 already pass. Maybe processing the deposits with the pre-Electra logic is actually equivalent, and we could just delete the code here that loops over pending_deposits?

[pawanjay176]

Yeah i'm okay with that. It seemed redundant to me initially but i just implemented it for completeness.
I'm fine with removing it

[pawanjay176]

Removed in d5ec058

[michaelsproul]

@pawanjay176 I've optimised and simplified part of the logic in PendingDepositsContext::new. We can avoid looking up the validator index twice, and we also don't need to handle the case for withdrawn validators that are also new, because that's impossible:

• 0af6b0f

[michaelsproul]

Pushed another commit which I think fixes a consensus bug 🌶️

• 21591a3

Let me know if you agree with my reasoning @pawanjay176. The EF tests pass with both versions of the code, which implies test coverage could be improved here.

[pawanjay176]

Great find 🙌 Turns out there was a spec test to catch this specific case, but it was written incorrectly. This has been fixed in the linked consensus specs PR above

[michaelsproul]

Nice! Thanks for following that up!

[michaelsproul]

Some of the tests seem to be failing due to builder stuff. Did you mean to include mock-builder changes in this PR?

[pawanjay176]

Ahh I didn't realise I pushed the mock builder changes to this PR. I'll revert them and fix the tests.
Will make a separate PR for the mock builder changes