-
Notifications
You must be signed in to change notification settings - Fork 26
Open Tree of Life APIs v3
This page just describes a proposal. The current version of the APIs is the one described on this page. This is intended as a cleaned-up version of this google-drive doc.
We plan on developing a treemachineLITE tool that will
act as the server for the tree_of_life/* calls mentioned
above.
Unlike previous versions of the API, this tool will accept
a specification of the tree structure and synthesis-related
metadata.
The previous version used a "graph of life" neo4j database
as its input; that (for all intents and purposes)
limited us to constructing the tree using neo4j.
We do not anticipate building an http-interface that would
allow clients to push a new a new version of the synthetic
tree.
However, we do want to document the interface, so that
other people can produce trees and use the treemachineLITE
software to serve tree_of_life queries.
We plan to write the treemachineLITE to read in a newick representation of the tree and a JSON file with any additional information about a node.
The tree structure is to be described in one file using the rules described below.
The JSON file for additional data contains the fields described below with the node IDs of the newick used as keys for node and edge data.
In v1.0 of the synthetic tree format the node labels are either OTT Ids or (if not OTT Ids) arbitrary
strings with no meaning.
This is a slight tweak designed to make it easier to construct the full tree from smaller analyses.
The only difference between this format and the v1.0 is that:
- the newick input can consist of multiple newicks. One newick will have the root of the tree labelled with an ID that occurs in no other newick. In all other newicks, the root of the newick will have label that is a tip ID in the "ancestral" newick. This indicates the grafting point for the newicks. No other IDs are allowed to be re-used across files.
-
nodeandedgeinformation in the JSON can be specified in multiple files. A parser simply takes the union of the information. It will illegal to have conflicting information about same node and edge in different JSON files. Typically a node or edge would only be described in one file, but it is also permitted to have some complementary information about the same node/edge spread across multiple files.
We are planning to implement a registry for node/path IDs. After that is implemented, we will be able to make the restriction that every node ID in the newick corresponds to the ID of this node in the registry.
For named taxa, this will be the OTT ID as in v1 and v2 So this change is just a change in the semantics of the labels of nodes that are not in OTT (from "meaningless label" to "registered ID").
Newick is a commonly supported, terse format for expressing tree structure, but it is weak in terms of expressing other information because the meaning of the node labels and branch length information is not specified by the standard. The New Hampshire Extended convention and the metacomments used by BEAST and associated tools rectify this via "hot comments". While these solutions work, they increase the size of the newick. For large trees, this makes handling the tree representation cumbersome, and is particularly galling for client code that is only interested in the tree representation.
NeXML if very nice for representing rich data, but also results in a very large representation for at tree of over 2 million leaves. The richness of NeXML's annotations relies heavily on the fact that the fundamental entities of the format have IDs that can serve as the target of an annotation.
The format obeys the rules of the standard newick format, but addes the following restrictions:
- Every node must have a label (hence "complete")
- each label fits the regex
[a-zA-Z0-9]+. In other words only numbers and roman alphabet are allowed. (hence "simple") - each label is unique (hence "id-labelled") in the context of the newick (not necessarily a globally unique ID).
- branch lengths are not included in the newick representation (and therefore, colons do not appear in the tree representation
The unique IDs can be used in accompanying data structures to uniquely refer to any node in the tree. The mandatory ID expands the size of the newicks somewhat, but requiring simple node labels makes it much easier to implement a validating parser.