Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wip: prototype node states graph #1186

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from
Draft

Conversation

jameshadfield
Copy link
Member

@jameshadfield jameshadfield commented Jul 16, 2020

Currently Auspice can convey metadata in two ways: via colouring of nodes and via a 2d layout of discrete trait values which are "geographical", i.e. those for which a lat/long value has been supplied by the dataset. There are metadata traits we wish to view which are not inherently geographical.

The PR contains a very early prototype version of auspice which can display "geo-resolutions" which are not backed by lat/longs. It is not optimised, leans heavily on code written for the Map component, and is missing lots of functionality. My idea is to display either the Map or a graph of states depending on whether lat/longs are available, but both are currently displayed for comparison.

Early feedback is very much welcome! Example dataset: https://auspice-show-state-tran-mdvh0q.herokuapp.com/testing/states?r=subclade_membership

image

@jameshadfield jameshadfield temporarily deployed to auspice-show-state-tran-mdvh0q July 16, 2020 06:30 Inactive
@jameshadfield jameshadfield force-pushed the show-state-transitions branch from 762dff7 to 4448daf Compare July 16, 2020 06:33
@jameshadfield jameshadfield temporarily deployed to auspice-show-state-tran-mdvh0q July 16, 2020 06:34 Inactive
@trvrb
Copy link
Member

trvrb commented Jul 16, 2020

Brilliant idea! I had thought about this for commonly reconstructed node states like "host" (https://mbio.asm.org/content/mbio/9/3/e00909-18/F6.large.jpg), but very clever that this works for "clade" as well.

The circular arrangement sometimes helps to pick out transition patterns: https://auspice-show-state-tran-mdvh0q.herokuapp.com/zika

It might make sense to limit the options to those that either have a geographic basis (with supplied lat/longs) or those that have internal node states labeled. Ie I'm not sure visualizations like https://auspice-show-state-tran-mdvh0q.herokuapp.com/testing/states (network layout by "country") are helpful.

In addition to circular arrangement you could imagine a force-directed layout option.

I would think it's more intuitive to replace the contents of the map panel with this "node states" or "transitions" visualization with a toggle in the sidebar, rather than having this be a separate panel, but I could be convinced otherwise (that having separate panels is better).

@jameshadfield
Copy link
Member Author

Thanks for the feedback @trvrb

It might make sense to limit the options to those that either have a geographic basis (with supplied lat/longs) or those that have internal node states labeled.

This prototype requires the dataset JSON to supply the values as the geo-resolutions options, my idea being to rename these as "spatial resolutions" and display either the map panel or this states panel depending on if lat-longs were defined for that resolution. (Happy for other suggestions tho.) You could even imagine the ability to define the spatial location of your state values akin to how we define the spatial location of countries by lat/long values.

In addition to circular arrangement you could imagine a force-directed layout option.

💯 yes! and with ability to drag demes etc (further down the track of course)

@trvrb
Copy link
Member

trvrb commented Jul 16, 2020

This prototype requires the dataset JSON to supply the values as the geo-resolutions options, my idea being to rename these as "spatial resolutions" and display either the map panel or this states panel depending on if lat-longs were defined for that resolution. (Happy for other suggestions tho.)

This makes sense to me. Good to have this be explicit. However, situations like https://auspice-show-state-tran-mdvh0q.herokuapp.com/WNV/NA make me think it would be useful to allow the network layout even if you have a trait with lat/long values. Something like lat/longs default to map view but allow a toggle to network view.

Includes setting & loading state from a URL query "showStateGraph" when the user could have chosen either for a particular resolution.
Previously it didn't make sense for a selected genotype to be a geo-resolution as there's no logical mapping between a nucleotide/aa and a lat-long. However for a force-directed state-map we do want to be able to visualise these.

This changes the behavior so that when a genotype is selected a new option appears in the available geo-resolutions. If the color-by changes to a non-genotype, then this option is removed. If the selected geo-res is the genotype, and the color-by changes, we either update the geo res to the new genotype color-by or switch to the default geo res.

This necessitated refactoring `getTraitFromNode` to handle genotype traits, and allowed us to get rid of the `getTipColorAttribute` function.
@jameshadfield jameshadfield temporarily deployed to auspice-show-state-tran-mdvh0q July 20, 2020 06:02 Inactive
@jameshadfield
Copy link
Member Author

Update: Now working using (partial) force-directed layout which is interactive (i.e. you can drag demes around). Main outstanding tasks before this can be considered for merge into master are to update appropriately when redux state changes and to improve the physics of the force-directed layout.

@trvrb
Copy link
Member

trvrb commented Jul 20, 2020

So cool! I love this direction. Given that force directed is not always going to be the best layout (and will often be finicky), I'd think it valuable to allow either a circular layout or a force-directed layout (and have a UI for this either in the sidebar or as tabs in the top-right of the panel). In terms of merge strategy I'd be okay merging with a circular layout to avoid the finicky improvements to force-directed (but open to other strategies).

Words here are tricky, but I'd think it slightly more intuitive to call this "Show network layout view" rather than "Show state layout view".

For the force-directed layout parameters, it looks like it might be better to just freely zoom the bounds rather than constrain with forces pinning nodes within the bounding box, eg https://auspice-show-state-tran-mdvh0q.herokuapp.com/zika?showStateMap results in every node along the bounding box, but I'd assuming you could remove this constraint and just zoom the camera out.

@rneher
Copy link
Member

rneher commented Jul 20, 2020

I like this as well! Essentially a 2-d histogram.

But I agree with Trevor that the force directed layout is not always sensible. This for example makes all circles fly away:

https://auspice-show-state-tran-mdvh0q.herokuapp.com/zika?c=gt-NS1_349&p=grid&showStateMap&transmissions=hide

But I think you can adjust some center gravity or similar to keep these circles in view. But arrange in a circle and make them draggable is probably useful.

This commit turns a proof-of-principle implementation into a working feature with acceptable performance. As props change (e.g. color-by, visibility) the visualization updates as needed. There are further performance improvements possible.

The main outstanding task is to improve the physics of the simulation.
Sets up d3 forces (and a custom force) to better position demes. Working reasonably well, but the performance when there are large numbers of transitions is unacceptable.
@jameshadfield jameshadfield temporarily deployed to auspice-show-state-tran-mdvh0q July 24, 2020 00:13 Inactive
@jameshadfield
Copy link
Member Author

Update: It's getting closer to release-worthy I think. The force layout is working pretty well for most use cases, but when there are lots of transmissions it's unacceptably slow and when dragging demes it can be annoying that they snap back so much. All of this is tweakable, and there are plenty of performance improvements which can be implemented. Additionally, there are a few necessities such as handling browser resizes and more testing. Feedback on the current force-layout would be helpful for setting the best parameters.

Here's WNV/NA

image

and zika

image

@rneher
Copy link
Member

rneher commented Jul 24, 2020

zika is very pretty :)

@trvrb trvrb temporarily deployed to auspice-show-state-tran-oz62dr January 13, 2021 01:32 Inactive
@trvrb
Copy link
Member

trvrb commented Jan 13, 2021

I'd like to bump this prototype for development and inclusion in master. I just played around with things again and I still think it's a brilliant direction. Here's the current clade_membership view for ncov/global:

clades

If we make a "constellations" trait with states like 417N,484K,501Y evolution of the virus between these eight genotypes would be described by connecting lines.

I really think this could help to outline genetic relationships between circulating variants.

It looks like the biggest issue is choosing robust parameters for the network view. One option here would be to give three values to click on in sidebar to toggle this between "strong", "medium" and "weak".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
experiment PRs which may never be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants