Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge #1594: Improve performance with better algorithms and caching o…
…f expensive computed values c3c10f1 wallet: implement optional cache validation (Matt Whitlock) 5bc7eb4 wallet: add persistent cache, mapping path->(priv, pub, script, addr) (Matt Whitlock) 01ec2a4 wallet: add _addr_map, paralleling _script_map (Matt Whitlock) 64f18bc get_imported_privkey_branch: use O(m+n) algorithm instead of O(m*n) (Matt Whitlock) 77f0194 wallet_utils: use new get_utxos_at_mixdepth method (Matt Whitlock) 184d76f wallet: add get_{balance,utxos}_at_mixdepth methods (Matt Whitlock) fc1e000 wallet_showutxos: use O(1) check for frozen instead of O(n) (Matt Whitlock) b58ac67 wallet: drop _get_addr_int_ext; replace with calls to get_new_addr (Matt Whitlock) 2c38a81 wallet: delete redundant get_script and get_addr methods (Matt Whitlock) 574c29e wallet: hoist get_script_from_path default impl into BaseWallet (Matt Whitlock) 8245271 wallet: avoid IndexError in _is_my_bip32_path (Matt Whitlock) 48aec83 wallet: remove a dead store in get_index_cache_and_increment (Matt Whitlock) Pull request description: **Note:** Reviewing each commit individually will make more sense than trying to review the combined diff. This PR implements several performance enhancements that take the CPU time to run `wallet-tool.py display` on my wallet down from ~44 minutes to ~11 seconds. The most significant gains come from replacing an **O**(_m_*_n_) algorithm in `get_imported_privkey_branch` with a semantically equivalent **O**(_m_+_n_) algorithm and from adding a persistent cache for computed private keys, public keys, scripts, and addresses. Below are some actual benchmarks on my wallet, which has 5 mixdepths, each having path indices reaching into the 4000s, and almost 700 imported private keys. * 673fbfb `origin/master` (baseline) ``` user 44m3.618s sys 0m6.375s ``` * 48aec83 `wallet`: remove a dead store in `get_index_cache_and_increment` * fbb681a `wallet`: add `get_{balance,utxos}_at_mixdepth` methods * 75a9703 `wallet_utils`: use new `get_utxos_at_mixdepth` method ``` user 42m14.464s sys 0m3.355s ``` * 84966e6 `wallet_showutxos`: use **O**(1) check for frozen instead of **O**(_n_) * 75c5a75 `get_imported_privkey_branch`: use **O**(_m_+_n_) algorithm instead of **O**(_m_*_n_) ``` user 5m0.045s sys 0m0.453s ``` * da8daf0 `wallet`: add `_addr_map`, paralleling `_script_map` ``` user 4m56.175s sys 0m0.423s ``` * d8aa1af `wallet`: add persistent cache, mapping path->(priv, pub, script, addr) ``` user 1m42.272s sys 0m0.471s ``` * After running another command to modify the wallet file so as to persist the cache, `wallet-tool.py display` now runs in: ``` user 0m11.141s sys 0m0.225s ``` ACKs for top commit: AdamISZ: tACK c3c10f1 Tree-SHA512: fdd20d436d8f16a1e4270011ad1ba4bf6393f876eb7413da30f75d5830249134911d5d93cab8051c0bf107c213d4cd46ba9614ae23eef4566f867ff1b912fc9b
- Loading branch information