MRG: add full column descriptions for gather and prefetch output

[ctb]

This PR adds full column descriptions for gather and prefetch to classifying-signatures.md. It also updates some other details in that document, including adding a link to the published Hera et al. paper in 2023.

See rendered docs!

Fixes #2812
Fixes #2367

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (9033d6d) 86.61% compared to head (40aae2a) 86.61%.

Additional details and impacted files

@@           Coverage Diff           @@
##           latest    #2954   +/-   ##
=======================================
  Coverage   86.61%   86.61%           
=======================================
  Files         135      135           
  Lines       15262    15262           
  Branches     2622     2622           
=======================================
  Hits        13219    13219           
  Misses       1743     1743           
  Partials      300      300           

Flag	Coverage Δ
hypothesis-py	25.83% <ø> (ø)
python	92.82% <ø> (ø)
rust	59.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ctb]

@sourmash-bio/devs ready for review and merge!

[ccbaumler]

What about a table instead of a list?

Gather column header	Type	Description
unique_intersect_bp	integer	Size of overlap between match and remaining query, estimated by multiplying the number of overlapping hashes by scaled. Rank/order dependent. Does not double count hashes.
intersect_bp	integer	Size of overlap between match and query, estimated by multiplying the number of overlapping hashes by scaled. Independent of rank order and will often double-count hashes.
f_orig_query	float	The fraction of the original query represented by this match. Approximates the fraction of metagenomic reads that will map to this genome.
f_match	float	The containment of the match in the query.
f_unique_to_query	float	The fraction of matching hashes (unweighted) that are unique to this query; rank dependent. Will sum to the fraction of total k-mers (unweighted) that were identified.
f_unique_weighted	float	The fraction of matching hashes (weighted by multiplicity) that are unique to this query. This will sum to the fraction of total weighted k-mers that were identified. Approximates the fraction of metagenomic reads that will map to this genome after all previous matches at lower (earlier) ranks are mapped.
average_abund	float	Mean abundance of the weighted hashes unique to the intersection. Empty if query does not have abundance. Rank dependent, does not double count.
median_abund	integer	Median abundance of the weighted hashes unique to the intersection. Empty if query has no abundance. Rank dependent, does not double count.
std_abund	float	Std deviation of the abundance of the hashes unique to the intersection. Empty if query has no abundance. Rank dependent, does not double count.
filename	string	Filename/location of the database from which the match was loaded.
name	string	Full sketch name of the match.
md5	string	Full md5sum of the match sketch.
f_match_orig	float	The fraction of the match in the full query. Rank independent.
gather_result_rank	float	Rank of this match in the results.
remaining_bp	integer	How many bp remain in the query after subtracting this match, estimated by multiplying remaining hashes by scaled.
query_filename	string	The filename from which the query was loaded.
query_name	string	The query sketch name.
query_md5	string	Truncated md5sum of the query sketch.
query_bp	integer	Estimated number of bp in the query, estimated by multiplying the sketch size by scaled.
ksize	integer	K-mer size for the sketches used in the comparison.
moltype	string	Molecule type of the comparison.
scaled	integer	Scaled value of the comparison.
query_n_hashes	integer	Number of hashes in the query sketch.
query_abundance	boolean	True if the query has abundance information; False otherwise.
query_containment_ani	float	ANI estimated from the query containment in the match.
match_containment_ani	float	ANI estimated from the match containment in the query.
average_containment_ani	float	ANI estimated from the average of the query and match containment.
max_containment_ani	float	ANI estimated from the max of the query and match containment.
potential_false_negative	boolean	True if the sketch size(s) were too small to give a reliable ANI estimate. False otherwise.
n_unique_weighted_found	integer	Sum of (abundance-weighted) hashes found in this rank.
sum_weighted_found	integer	Sum of the hashes x abundance found thus far, i.e., running total of n_unique_weighted_found. The last value divided by total_weighted_hashes will equal the total fraction of (weighted) k-mers identified.
total_weighted_hashes	integer	Sum of hashes x abundance for the entire dataset. Constant value.

Prefetch column header	Type	Description
intersect_bp	integer	Size of overlap between match and original query, estimated by multiplying the number of overlapping hashes by scaled.
jaccard	float	Jaccard similarity of the two sketches.
max_containment	float	Max of f_query_match and f_match_query.
f_query_match	float	The fraction of the query contained by the match.
f_match_query	float	The fraction of the match contained by the query.
match_filename	string	Filename the match sketch was loaded from.
match_name	string	Full name of the match sketch.
match_md5	string	Truncated md5sum of match sketch (8 char).
match_bp	integer	Size of match, estimated by multiplying the sketch size by scaled.
query_filename	string	Filename the query sketch was loaded from.
query_name	string	Full name of the query sketch.
query_md5	string	Truncated md5sum of query sketch (8 char).
query_bp	integer	Size of query, estimated by multiplying the sketch size by scaled.
ksize	integer	K-mer size for the sketches used in the comparison.
moltype	string	Molecule type of the sketches.
scaled	integer	Scaled value at which the comparison was done.
query_n_hashes	integer	Number of hashes in the query.
query_abundance	integer	Median hash abundance in the sketch, if available.
query_containment_ani	float	ANI estimated from the query containment in the match.
match_containment_ani	float	ANI estimated from the match containment in the query.
average_containment_ani	float	ANI estimated from the average of the query and match containment.
max_containment_ani	float	ANI estimated from the max containment between query/match.
potential_false_negative	boolean	True if the sketch size(s) were too small to give a reliable ANI estimate. False if ANI estimate is reliable.

[ctb]

hmm, I like the table... I'm worried that updating it will be annoying. I think on it and maybe try it out!

[ctb]

columns work great, thanks @ccbaumler! Updated & will merge sometime after the tests pass, unless someone drops by with a new comment first ;)

[ccbaumler]

LGTM