docs: clarify chain upgrade procedure

Follow-up to #4097. We performed a chain upgrade on Penumbra
for v0.71.0. Based on feedback from the community validators, we've
refined the procedure somewhat. Substantive changes to the docs are as
follows:

  * backup the entire pd & cometbft data dirs, as a precaution
  * upgrade to new pd early, prior to running export, for simplicity's
    sake
  * only move the exported rocksdb directory, which allows preservation
    of other dirs in pd state, e.g. acme tls info
  * do not reset priv validator state
  * amend custom find command to preserve `tx_index.db`

Not yet included in these docs are specifics about the upcoming release;
we'll add those soon.

Finishes and therefore closes #1812.

docs/guide/src/node/pd/chain-upgrade.md

@@ -45,15 +45,14 @@ See an example below for how to supply this value when performing the migration.
 
 ## Performing a chain upgrade
 
-The following steps assume that `pd` is using the default home directory of `~/.penumbra/testnet_data/node0/pd`.
-If your instance is using a different directory, update the paths accordingly.
+The following steps assume that your node uses the default directory of `~/.penumbra/testnet_data/node0/`
+to store state for both `pd` and `cometbft`. If your instance is using a different directory, update the paths accordingly.
 
 1. Stop both `pd` and `cometbft`. Depending on how you run Penumbra, this could mean `sudo systemctl stop penumbra cometbft`.
-2. Using the same version of `pd` that was running when the chain halted, prepare an export directory:
+2. Back up the existing node state, as a precaution: `tar -cf ~/.penumbra/testnet_data/node0-state-backup.tar ~/.penumbra/testnet_data/node0`
+3. Download the latest version of `pd` and install it. Run `pd --version` and confirm you see `{{ #include ../../penumbra_version.md }}` before proceeding.
+4. Prepare an export directory:
    `pd export --home ~/.penumbra/testnet_data/node0/pd --export-directory ~/.penumbra/testnet_data/node0/pd-exported-state`
-3. Back up the historical state directory: `mv ~/.penumbra/testnet_data/node0/pd ~/.penumbra/testnet_data/node0/pd-state-backup`
-4. Download the latest version of `pd` and install it. Run `pd --version` and confirm you see `{{ #include ../../penumbra_version.md }}` before proceeding.
-
 <!--
 An example log message emitted by `pd migrate` without providing `--genesis-start`:
 
@@ -64,10 +63,14 @@ and provide that value in the documentation. Or should we just pick a genesis st
 -->
 5. Apply the migration: `pd migrate --genesis-start "{{ #include ../../upgrade_genesis_time_70_71.md }}" --target-directory ~/.penumbra/testnet_data/node0/pd-exported-state/ --migrate-archive ~/.penumbra/testnet_data/node0/pd-migrated-state-{{ #include ../../penumbra_version.md }}.tar.gz`.
    You must use that precise genesis time, otherwise your node will not be able to reach consensus with the rest of the network.
-6. Move the migrated state into place: `mkdir ~/.penumbra/testnet_data/node0/pd && mv ~/.penumbra/testnet_data/node0/pd-exported-state/rocksdb ~/.penumbra/testnet_data/node0/pd/`
-7. Move the upgrade cometbft state into place: `cp ~/.penumbra/testnet_data/node0/pd-exported-state/genesis.json ~/.penumbra/testnet_data/node0/cometbft/config/genesis.json
-   && cp ~/.penumbra/testnet_data/node0/pd-exported-state/priv_validator_state.json ~/.penumbra/testnet_data/node0/cometbft/data/priv_validator_state.json`
-8. Then we clean up the old CometBFT state: `find ~/.penumbra/testnet_data/node0/cometbft/data/ -mindepth 1 -maxdepth 1 -type d -exec rm -r {} +`
+6. Move the migrated state into place: `rm -r ~/.penumbra/testnet_data/node0/pd/rocksdb && mv ~/.penumbra/testnet_data/node0/pd-exported-state/rocksdb ~/.penumbra/testnet_data/node0/pd/`
+7. Copy the new genesis into place: `cp ~/.penumbra/testnet_data/node0/pd-exported-state/genesis.json ~/.penumbra/testnet_data/node0/cometbft/config/genesis.json`
+8. Clean up the old CometBFT state: `find ~/.penumbra/testnet_data/node0/cometbft/data/ -mindepth 1 -maxdepth 1 -type d -and -not -name tx_index.db -exec rm -r {} +`
+
+<!--
+N.B. We use an ugly ad-hoc find command rather than `cometbft reset-state` because we want to preserve the `tx_index.db` directory.
+Doing so will allow CometBFT to reference historical transactions behind the upgrade boundary.
+-->
 
 Finally, restart the node, e.g. `sudo systemctl restart penumbra cometbft`. Check the logs, and you should see the chain progressing
 past the halt height `n`.