Move more logic in blockstore_processor behind allow_dead_slots flag

A flag in ProcessOptions can be set to allow dead slots. If the flag is
not set and a block is marked dead, fetching the entries will fail as
the entry fetch method internally checks if the slot is dead.

Within process_next_slots(), new Banks are created as replay progresses
through slots. If allow_dead_slots=false and a dead slot is loaded,
replay of the slot will error. That error is handled and results in the
Bank being removed from BankForks.

Instead of allowing the extra work to proceed, check if new slots to
replay are dead (and allow_dead_slots=false) BEFORE the creation of a
new Bank.

ledger/src/blockstore_processor.rs

@@ -1725,15 +1725,21 @@ fn process_next_slots(
     blockstore: &Blockstore,
     leader_schedule_cache: &LeaderScheduleCache,
     pending_slots: &mut Vec<(SlotMeta, Bank, Hash)>,
-    halt_at_slot: Option<Slot>,
+    opts: &ProcessOptions,
 ) -> result::Result<(), BlockstoreProcessorError> {
     if meta.next_slots.is_empty() {
         return Ok(());
     }
 
     // This is a fork point if there are multiple children, create a new child bank for each fork
     for next_slot in &meta.next_slots {
-        if halt_at_slot.is_some_and(|halt_at_slot| *next_slot > halt_at_slot) {
+        if opts
+            .halt_at_slot
+            .is_some_and(|halt_at_slot| *next_slot > halt_at_slot)
+        {
+            continue;
+        }
+        if !opts.allow_dead_slots && blockstore.is_dead(*next_slot) {
             continue;
         }
 
@@ -1812,7 +1818,7 @@ fn load_frozen_forks(
         blockstore,
         leader_schedule_cache,
         &mut pending_slots,
-        opts.halt_at_slot,
+        opts,
     )?;
 
     let on_halt_store_hash_raw_data_for_debug = opts.on_halt_store_hash_raw_data_for_debug;
@@ -1991,7 +1997,7 @@ fn load_frozen_forks(
                 blockstore,
                 leader_schedule_cache,
                 &mut pending_slots,
-                opts.halt_at_slot,
+                opts,
             )?;
         }
     } else if on_halt_store_hash_raw_data_for_debug {