WIP: Add scripts/pairwise_to_matrix/ helper script

[mr-eyes]

Description

This pull request adds a new 'scripts/' directory to the Branchwater project. The directory is dedicated to stand-alone scripts designed to post-process the output generated by various Branchwater commands.

Adding scripts/pairwise_to_matrix

This script efficiently converts pairwise command outputs into sparse matrices and enables their export to TSV format. The script uses Dask, so it's engineered for large-scale data processing.

[ctb]

thanks!!

The downside to this approach is that scripts/ isn't installed by Python currently ;). So more work needed there...

[ctb]

and, as noted in the docs, lots of tricky dependencies - esp h5py. I suspect this will work well as a separate plugin!

[ctb]

a dumb, and presumably much less efficient, version of this functionality is available in https://github.com/sourmash-bio/sourmash_plugin_betterplot/ as pairwise_to_matrix.

[ctb]

a dumb, and presumably much less efficient, version of this functionality is available in https://github.com/sourmash-bio/sourmash_plugin_betterplot/ as pairwise_to_matrix.

To resolve this PR, we should benchmark it against pairwise_to_matrix in betterplot. If it is in fact much, much less memory intensive and/or faster, we should keep it and consider merging (or incorporating in a different plugin). If it's reasonably comparable, I would argue for instead closing it and putting effort into testing the pairwise_to_matrix code :).