Skip to content

Commit

Permalink
Merge #1594: Improve performance with better algorithms and caching o…
Browse files Browse the repository at this point in the history
…f expensive computed values

c3c10f1 wallet: implement optional cache validation (Matt Whitlock)
5bc7eb4 wallet: add persistent cache, mapping path->(priv, pub, script, addr) (Matt Whitlock)
01ec2a4 wallet: add _addr_map, paralleling _script_map (Matt Whitlock)
64f18bc get_imported_privkey_branch: use O(m+n) algorithm instead of O(m*n) (Matt Whitlock)
77f0194 wallet_utils: use new get_utxos_at_mixdepth method (Matt Whitlock)
184d76f wallet: add get_{balance,utxos}_at_mixdepth methods (Matt Whitlock)
fc1e000 wallet_showutxos: use O(1) check for frozen instead of O(n) (Matt Whitlock)
b58ac67 wallet: drop _get_addr_int_ext; replace with calls to get_new_addr (Matt Whitlock)
2c38a81 wallet: delete redundant get_script and get_addr methods (Matt Whitlock)
574c29e wallet: hoist get_script_from_path default impl into BaseWallet (Matt Whitlock)
8245271 wallet: avoid IndexError in _is_my_bip32_path (Matt Whitlock)
48aec83 wallet: remove a dead store in get_index_cache_and_increment (Matt Whitlock)

Pull request description:

  **Note:** Reviewing each commit individually will make more sense than trying to review the combined diff.

  This PR implements several performance enhancements that take the CPU time to run `wallet-tool.py display` on my wallet down from ~44 minutes to ~11 seconds.

  The most significant gains come from replacing an **O**(_m_*_n_) algorithm in `get_imported_privkey_branch` with a semantically equivalent **O**(_m_+_n_) algorithm and from adding a persistent cache for computed private keys, public keys, scripts, and addresses.

  Below are some actual benchmarks on my wallet, which has 5 mixdepths, each having path indices reaching into the 4000s, and almost 700 imported private keys.

  * 673fbfb `origin/master` (baseline)
      ```
      user    44m3.618s
      sys     0m6.375s
      ```
  * 48aec83 `wallet`: remove a dead store in `get_index_cache_and_increment`
  * fbb681a `wallet`: add `get_{balance,utxos}_at_mixdepth` methods
  * 75a9703 `wallet_utils`: use new `get_utxos_at_mixdepth` method
      ```
      user    42m14.464s
      sys     0m3.355s
      ```
  * 84966e6 `wallet_showutxos`: use **O**(1) check for frozen instead of **O**(_n_)
  * 75c5a75 `get_imported_privkey_branch`: use **O**(_m_+_n_) algorithm instead of **O**(_m_*_n_)
      ```
      user    5m0.045s
      sys     0m0.453s
      ```
  * da8daf0 `wallet`: add `_addr_map`, paralleling `_script_map`
      ```
      user    4m56.175s
      sys     0m0.423s
      ```
  * d8aa1af `wallet`: add persistent cache, mapping path->(priv, pub, script, addr)
      ```
      user    1m42.272s
      sys     0m0.471s
      ```
  * After running another command to modify the wallet file so as to persist the cache, `wallet-tool.py display` now runs in:
      ```
      user    0m11.141s
      sys     0m0.225s
      ```

ACKs for top commit:
  AdamISZ:
    tACK c3c10f1

Tree-SHA512: fdd20d436d8f16a1e4270011ad1ba4bf6393f876eb7413da30f75d5830249134911d5d93cab8051c0bf107c213d4cd46ba9614ae23eef4566f867ff1b912fc9b
  • Loading branch information
AdamISZ committed Dec 1, 2023
2 parents f3dd99d + c3c10f1 commit 2d3e90a
Show file tree
Hide file tree
Showing 5 changed files with 405 additions and 213 deletions.
Loading

0 comments on commit 2d3e90a

Please sign in to comment.