Skip to content

Commit

Permalink
Adds migrations to restore currupted staking ledgers in Polkadot and …
Browse files Browse the repository at this point in the history 
…Kusama (#447)

Note: for more details on the corrupted ledgers issue and recovery steps
check https://hackmd.io/m_h9DRutSZaUqCwM9tqZ3g?view.

This PR adds a migration in Polkadot and Kusama runtimes to recover the
current corrupted ledgers in Polkadot and Kusama. A migration consists
of:

1. Call into `pallet_staking::Pallet::<T>::restore_ledger` for each of
the "whitelisted" stashes as `Root` origin.
2. Performs a check that ensures the restored ledger's stake does not
overflow the current stash's free balance. If that's the case, force
unstake the ledger. This check is currently missing in
polkadot-sdk/pallet-staking ([PR with patch
here](paritytech/polkadot-sdk#5066)).

The reason to restore the corrupted ledgers as migrations implemented in
the fellowship runtimes is twofold:

1. The call to `pallet_staking::Pallet::<T>::restore_ledger` and check +
`force_unstake` must be done atomically (thus a ledger can't be safely
restored with a set of two distinct extrinsic calls, so it's not
possible to use referenda to this fx).
2. To speed up the whole process and avoid having to wait for 1. merge
and releases of paritytech/polkadot-sdk#5066 and
2. referenda to call into `Call::restore_ledger` for both Polkadot and
Kusama.

Alternatively, we could add a new temporary pallet directly in the
fellowship runtime which would expose an extrinsic to restore the
ledgers and perform the extra missing check. See this [PR as an
example](gpestana#2).

---
- [x] on-runtime-upgrade tests against Polkadot and Kusama
- [x] staking try-state checks passing after all migrations.
  • Loading branch information
@gpestana
gpestana authored Oct 2, 2024
1 parent a1c5878 commit 065c332
Show file tree
Hide file tree
Showing 4 changed files with 291 additions and 0 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

- Chain-spec generator: propagate the `on_chain_release_build` feature to the chain-spec generator. Without this the live/genesis chain-specs contain a wrongly-configured WASM blob ([polkadot-fellows/runtimes#450](https://github.com/polkadot-fellows/runtimes/pull/450)).
- Adds a migration to the Polkadot Coretime chain to fix an issue from the initial Coretime migration. ([polkadot-fellows/runtimes#458](https://github.com/polkadot-fellows/runtimes/pull/458))
- Adds migrations to restore currupted staking ledgers in Polkadot and Kusama ([polkadot-fellows/runtimes#447](https://github.com/polkadot-fellows/runtimes/pull/447))

### Added

Expand Down
143 changes: 143 additions & 0 deletions relay/kusama/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1817,12 +1817,155 @@ pub mod migrations {
pallet_staking::migrations::v15::MigrateV14ToV15<Runtime>,
parachains_inclusion::migration::MigrateToV1<Runtime>,
parachains_assigner_on_demand::migration::MigrateV0ToV1<Runtime>,
restore_corrupted_ledgers::Migrate<Runtime>,
);

/// Migrations/checks that do not need to be versioned and can run on every update.
pub type Permanent = (pallet_xcm::migration::MigrateToLatestXcmVersion<Runtime>,);
}

/// Migration to fix current corrupted staking ledgers in Kusama.
///
/// It consists of:
/// * Call into `pallet_staking::Pallet::<T>::restore_ledger` with:
/// * Root origin;
/// * Default `None` paramters.
/// * Forces unstake of recovered ledger if the final restored ledger has higher stake than the
/// stash's free balance.
///
/// The stashes associated with corrupted ledgers that will be "migrated" are set in
/// [`CorruptedStashes`].
///
/// For more details about the corrupt ledgers issue, recovery and which stashes to migrate, check
/// <https://hackmd.io/m_h9DRutSZaUqCwM9tqZ3g?view>.
pub(crate) mod restore_corrupted_ledgers {
use super::*;

use frame_support::traits::{Currency, OnRuntimeUpgrade};
use frame_system::RawOrigin;

use pallet_staking::WeightInfo;
use sp_staking::StakingAccount;

parameter_types! {
pub CorruptedStashes: Vec<AccountId> = vec![
// stash account ESGsxFePah1qb96ooTU4QJMxMKUG7NZvgTig3eJxP9f3wLa
hex_literal::hex!["52559f2c7324385aade778eca4d7837c7492d92ee79b66d6b416373066869d2e"].into(),
// stash account DggTJdwWEbPS4gERc3SRQL4heQufMeayrZGDpjHNC1iEiui
hex_literal::hex!["31162f413661f3f5351169299728ab6139725696ac6f98db9665e8b76d73d300"].into(),
// stash account Du2LiHk1D1kAoaQ8wsx5jiNEG5CNRQEg6xME5iYtGkeQAJP
hex_literal::hex!["3a8012a52ec2715e711b1811f87684fe6646fc97a276043da7e75cd6a6516d29"].into(),
];
}

pub struct Migrate<T>(sp_std::marker::PhantomData<T>);
impl<T: pallet_staking::Config> OnRuntimeUpgrade for Migrate<T> {
fn on_runtime_upgrade() -> Weight {
let mut total_weight: Weight = Default::default();
let mut ok_migration = 0;
let mut err_migration = 0;

for stash in CorruptedStashes::get().into_iter() {
let stash_account: T::AccountId = if let Ok(stash_account) =
T::AccountId::decode(&mut &Into::<[u8; 32]>::into(stash.clone())[..])
{
stash_account
} else {
log::error!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: error converting account {:?}. skipping.",
stash.clone(),
);
err_migration += 1;
continue
};

// restore currupted ledger.
match pallet_staking::Pallet::<T>::restore_ledger(
RawOrigin::Root.into(),
stash_account.clone(),
None,
None,
None,
) {
Ok(_) => (), // proceed.
Err(err) => {
// note: after first migration run, restoring ledger will fail with
// `staking::pallet::Error::<T>CannotRestoreLedger`.
log::error!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: error restoring ledger {:?}, unexpected (unless running try-state idempotency round).",
err
);
continue
},
};

// check if restored ledger total is higher than the stash's free balance. If
// that's the case, force unstake the ledger.
let weight = if let Ok(ledger) = pallet_staking::Pallet::<T>::ledger(
StakingAccount::Stash(stash_account.clone()),
) {
// force unstake the ledger.
if ledger.total > T::Currency::free_balance(&stash_account) {
let slashing_spans = 10; // default slashing spans for migration.
let _ = pallet_staking::Pallet::<T>::force_unstake(
RawOrigin::Root.into(),
stash_account.clone(),
slashing_spans,
)
.map_err(|err| {
log::error!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: error force unstaking ledger, unexpected. {:?}",
err
);
err_migration += 1;
err
});

log::info!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: ledger of {:?} restored (with force unstake).",
stash_account,
);
ok_migration += 1;

<T::WeightInfo>::restore_ledger()
.saturating_add(<T::WeightInfo>::force_unstake(slashing_spans))
} else {
log::info!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: ledger of {:?} restored.",
stash,
);
ok_migration += 1;

<T::WeightInfo>::restore_ledger()
}
} else {
log::error!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: ledger does not exist, unexpected."
);
err_migration += 1;
<T::WeightInfo>::restore_ledger()
};

total_weight.saturating_accrue(weight);
}

log::info!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: done. success: {}, error: {}",
ok_migration, err_migration
);

total_weight
}
}
}

/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
Expand Down
1 change: 1 addition & 0 deletions relay/polkadot/Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ pallet-election-provider-support-benchmarking = { optional = true, workspace = t
pallet-offences-benchmarking = { optional = true, workspace = true }
pallet-session-benchmarking = { optional = true, workspace = true }
pallet-nomination-pools-benchmarking = { optional = true, workspace = true }
hex-literal = { workspace = true }

polkadot-runtime-common = { workspace = true }
runtime-parachains = { workspace = true }
Expand Down
146 changes: 146 additions & 0 deletions relay/polkadot/src/lib.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2056,12 +2056,158 @@ pub mod migrations {
GetLegacyLeaseImpl,
>,
CancelAuctions,
restore_corrupted_ledgers::Migrate<Runtime>,
);

/// Migrations/checks that do not need to be versioned and can run on every update.
pub type Permanent = (pallet_xcm::migration::MigrateToLatestXcmVersion<Runtime>,);
}

/// Migration to fix current corrupted staking ledgers in Polkadot.
///
/// It consists of:
/// * Call into `pallet_staking::Pallet::<T>::restore_ledger` with:
/// * Root origin;
/// * Default `None` paramters.
/// * Forces unstake of recovered ledger if the final restored ledger has higher stake than the
/// stash's free balance.
///
/// The stashes associated with corrupted ledgers that will be "migrated" are set in
/// [`CorruptedStashes`].
///
/// For more details about the corrupt ledgers issue, recovery and which stashes to migrate, check
/// <https://hackmd.io/m_h9DRutSZaUqCwM9tqZ3g?view>.
pub(crate) mod restore_corrupted_ledgers {
use super::*;

use frame_support::traits::Currency;
use frame_system::RawOrigin;

use pallet_staking::WeightInfo;
use sp_staking::StakingAccount;

parameter_types! {
pub CorruptedStashes: Vec<AccountId> = vec![
// stash account 138fZsNu67JFtiiWc1eWK2Ev5jCYT6ZirZM288tf99CUHk8K
hex_literal::hex!["5e510306a89f40e5520ae46adcc7a4a1bbacf27c86c163b0691bbbd7b5ef9c10"].into(),
// stash account 14kwUJW6rtjTVW3RusMecvTfDqjEMAt8W159jAGBJqPrwwvC
hex_literal::hex!["a6379e16c5dab15e384c71024e3c6667356a5487127c291e61eed3d8d6b335dd"].into(),
// stash account 13SvkXXNbFJ74pHDrkEnUw6AE8TVkLRRkUm2CMXsQtd4ibwq
hex_literal::hex!["6c3e8acb9225c2a6d22539e2c268c8721b016be1558b4aad4bed220dfbf01fea"].into(),
// stash account 12YcbjN5cvqM63oK7WMhNtpTQhtCrrUr4ntzqqrJ4EijvDE8
hex_literal::hex!["4458ad5f0c082da64610607beb9d3164a77f1ef7964b7871c1de182ea7213783"].into(),
];
}

pub struct Migrate<T>(sp_std::marker::PhantomData<T>);
impl<T: pallet_staking::Config> OnRuntimeUpgrade for Migrate<T> {
fn on_runtime_upgrade() -> Weight {
let mut total_weight: Weight = Default::default();
let mut ok_migration = 0;
let mut err_migration = 0;

for stash in CorruptedStashes::get().into_iter() {
let stash_account: T::AccountId = if let Ok(stash_account) =
T::AccountId::decode(&mut &Into::<[u8; 32]>::into(stash.clone())[..])
{
stash_account
} else {
log::error!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: error converting account {:?}. skipping.",
stash.clone(),
);
err_migration += 1;
continue
};

// restore currupted ledger.
match pallet_staking::Pallet::<T>::restore_ledger(
RawOrigin::Root.into(),
stash_account.clone(),
None,
None,
None,
) {
Ok(_) => (), // proceed.
Err(err) => {
// note: after first migration run, restoring ledger will fail with
// `staking::pallet::Error::<T>CannotRestoreLedger`.
log::error!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: error restoring ledger {:?}, unexpected (unless running try-state idempotency round).",
err
);
continue
},
};

// check if restored ledger total is higher than the stash's free balance. If
// that's the case, force unstake the ledger.
let weight = if let Ok(ledger) = pallet_staking::Pallet::<T>::ledger(
StakingAccount::Stash(stash_account.clone()),
) {
// force unstake the ledger.
if ledger.total > T::Currency::free_balance(&stash_account) {
let slashing_spans = 10; // default slashing spans for migration.
let _ = pallet_staking::Pallet::<T>::force_unstake(
RawOrigin::Root.into(),
stash_account.clone(),
slashing_spans,
)
.map_err(|err| {
log::error!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: error force unstaking ledger, unexpected. {:?}",
err
);
err_migration += 1;
err
});

log::info!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: ledger of {:?} restored (with force unstake).",
stash_account,
);
ok_migration += 1;

<T::WeightInfo>::restore_ledger()
.saturating_add(<T::WeightInfo>::force_unstake(slashing_spans))
} else {
log::info!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: ledger of {:?} restored.",
stash,
);
ok_migration += 1;

<T::WeightInfo>::restore_ledger()
}
} else {
log::error!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: ledger does not exist, unexpected."
);
err_migration += 1;
<T::WeightInfo>::restore_ledger()
};

total_weight.saturating_accrue(weight);
}

log::info!(
target: LOG_TARGET,
"migrations::corrupted_ledgers: done. success: {}, error: {}",
ok_migration,
err_migration
);

total_weight
}
}
}

/// Unchecked extrinsic type as expected by this runtime.
pub type UncheckedExtrinsic =
generic::UncheckedExtrinsic<Address, RuntimeCall, Signature, SignedExtra>;
Expand Down

0 comments on commit 065c332

Please sign in to comment.