feat: use many tasks to order streams and discover undelivered events at startup

[dav1do]

After an IPFS migration, we have to review all the data in the database to make sure we have the complete stream history before we can send events out of the API. We originally kept it simple and would read events and process them, then repeat in a single task. This was taking far too long on large datasets (e.g. 100s of GBs). Now we spawn multiple tasks to read the events from the database, and they send events over a channel to the ordering task (like we do during normal operation). This task was also modified to spawn multiple tasks to process events by stream and order them. Both changes appeared necessary during testing as one side would waiting on the other. This allows us to keep a solid rate of processing going and we've seen a substantial improvement in runtime (~60-100x faster).

On the discovery side: At startup, we spawn 16 tasks to read batches from the database. The number of events read each time was reduced to 250, as 1000 was taking seconds. The values are slightly arbitrary but this seemed like a "fast enough" choice during testing (the goal is simply to keep the channel full). The event data is partitioned using (rowid % number_tasks) = task_number so we don't have to do anything clever to split the data into batches up front. Each task starts from the beginning and pick up any events that have been missed. Once it finishes, the subsequent runs are fast, so we spawn the tasks regardless of whether they're needed.

On the ordering side a few changes were made. First, the channel size was reduced to 10000 (the previous value was far too large) and we try to empty it before doing any ordering since we have more tasks to process the set, and any events found may avoid database reads if they're for the same stream. Once events are grouped by stream, we split the streams into batches and spawn 1-16 tasks to process each batch. This processing has cpu bound work, but also requires database reads so multiple tasks have been beneficial. The tasks then send their ordered data back to the manager, which handles writing to the database. During testing, I made a change to remove a RO connection from the pool (and allow it to grow afterward) for each of these tasks. It didn't seem to make an obvious difference, but it may be useful to revisit.