add compact() API and auto-rebuild indexes

[staltz]

Context: #306 and ssbc/jitdb#199

This PR adds the compact() ("async") API to ssb-db2 which just triggers async-append-only-log compact(). When it's done, it will rebuild all leveldb indexes from scratch, rebuild private indexes, and partially rebuild jitdb indexes. This is all done in a way that is crash-resistant, capable of resuming where it had stopped. And compaction is guaranteed to not happen concurrently with log.stream that builds indexes, or any jitdb query.

[staltz]

There's an error in the test compaction resumes automatically after a crash:

RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 65534. Received 77082
    at new NodeError (internal/errors.js:322:7)
    at boundsError (internal/buffer.js:85:9)
    at Buffer.readUInt16LE (internal/buffer.js:242:5)
    at Object.readDataLength (/home/runner/work/ssb-db2/ssb-db2/node_modules/async-append-only-log/record.js:23:19)
    at Object.getDataNextOffset (/home/runner/work/ssb-db2/ssb-db2/node_modules/async-append-only-log/index.js:234:31)
    at Stream._handleBlock (/home/runner/work/ssb-db2/ssb-db2/node_modules/async-append-only-log/stream.js:89:37)
    at Stream._resumeCallback (/home/runner/work/ssb-db2/ssb-db2/node_modules/async-append-only-log/stream.js:145:27)
    at Object.getBlock (/home/runner/work/ssb-db2/ssb-db2/node_modules/async-append-only-log/index.js:201:7)
    at Stream._resume (/home/runner/work/ssb-db2/ssb-db2/node_modules/async-append-only-log/stream.js:136:12)
    at Stream._next (/home/runner/work/ssb-db2/ssb-db2/node_modules/looper/index.js:11:9) {
  code: 'ERR_OUT_OF_RANGE'
}

It happens mostly in CI, hard to reproduce locally. Have to solve this crash before the PR can be said to be ready.

[staltz]

I fixed this error, it was about updateIndexes() (and a log.stream()) happening concurrently with compaction.

When running compact() in ssb-db2, it checks whether there is any active updateIndexes() going on, postponing the compact if there are.

However, if we are resuming from a crash, then compaction is started automatically by async-append-only-log, and then updateIndexes() is also started automatically in ssb-db2. So I fixed this by adding an if condition in updateIndexes() which checks whether compaction is ongoing. This works because AAOL resumes compaction synchronously, while updateIndexes() is always started asynchronously (it waits for a promise to load, so we know that it happens later).

[staltz]

@arj03 Although this is the final task for ssb-db2, I have still 2 TODOs:

• EBT: we need to update or delete some entries after we delete msgs from the log. To keep it simple, let's suppose we only use sbot.db.deleteFeed and thus we delete that whole feedId from EBT too
• What about browser compatibility for AAOL/compaction.js and DB2 post-compact files?

[arj03]

This was something I used in browser core for testing. I don't mind removing this. It's a thin wrapper around indexes reset anyway.

[staltz]

Do you want to bring it back?

[arj03]

Nice work 🚀

[staltz]

Feels crazy to finally merge this, but here we go!

[github-actions]

Benchmark results

Part	Duration
add 1000 elements	499ms
add 1000 private box1 elements	1339ms
unbox 1000 private box1 elements first run	147ms
unbox 1000 private box1 elements second run	111ms
add 1000 private box1 elements	1364ms
query 1000 elements first run	40ms
query 1000 elements second run	27ms
add 1000 private box2 elements	823ms
unbox 1000 private box2 elements first run	429ms
unbox 1000 private box2 elements second run	441ms
Migrate (+db1)	14870ms
Migrate (alone)	4962ms
Migrate (+db1 +db2)	11202ms
Migrate (+db2)	7505ms
Migrate continuation (+db2)	1090ms
Memory usage without indexes	754.36 MB = 27.38 MB + etc
Initial indexing	705ms
Initial indexing maxCpu=86	4780ms
Initial indexing compat	1197ms
Two indexes updating concurrently	1060ms
key one initial	53ms
key two	5ms
key one again	0ms
reboot and key one again	51ms
latest root posts	660ms
latest posts	37ms
votes one initial	599ms
votes again	1ms
hasRoot	426ms
hasRoot again	0ms
author one posts	357ms
author two posts	19ms
dedicated author one posts	426ms
dedicated author one posts again	0ms
Maximum memory usage	811.09 MB = 45.74 MB + etc
Indexes folder size	9.97mb