Feature: benchmark memory usage with jemalloc

[romanz]

Following https://twitter.com/dpc_pw/status/1856972589140152700

[romanz]

@dpc would it please be possible to share the commits being used in https://twitter.com/dpc_pw/status/1857090601042198921?

[dpc]

I'm still working and investigating, but I'm currently testing: fedimint/fedimint#6354

[romanz]

When running with jemalloc (#1115), there is a ~10% memory usage improvement (right-side plot vs. left-side):

Tested with Electrum client loading the 1BitcoinEaterAddressDontSendf59kuE address (with >3.8k txs).

[antonilol]

10% seems far off from only using half the memory, sure, electrs itself also uses memory, but does #1115 also change the allocator used by rocksdb? Setting the global allocator in Rust will use it for Rust's heap allocated data structures (Vec, HashMap, BTreeMap, etc), but i am unsure about if this will cause rocksdb to also use it. (see facebook/rocksdb#12364, rust-rocksdb/rust-rocksdb#863)

[dpc]

@antonilol I think you're right and actually enabling jemalloc for rocksdb requires some work. In Fedimint it seems we're hitting some weird patterns around iterator usage on the Rust side where jemalloc helps significantly. I need to investigate enabling jemalloc for rocksdb itself and will share and follow this thread with anyone else trying the same thing.

[antonilol]

Runtime performance should also be measured, if memory usage drops by 50% but electrs gets significantly slower (at indexing or lookups), people should be able to make the decision about this tradeoff themselves (at compile time is fine, for me at least).

[dpc]

BTW. I have enabled jemalloc on rocksdb itself in Fedimint and we're seeing much better memory usage now. It's kind of hard to quantify, as it's a long running process that was accumulating memory asymptotically over weeks, but the difference is clearly visible.