Skip to content

Commit

Permalink
Seperated property_ranges query parameter from the partial data.
Browse files Browse the repository at this point in the history
  • Loading branch information
@JPBergsma @ml-evs
JPBergsma authored and ml-evs committed Dec 19, 2023
1 parent 93ecd0b commit a84c72e
Showing 1 changed file with 67 additions and 32 deletions.
99 changes: 67 additions & 32 deletions optimade.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -459,41 +459,73 @@ The default partial data format is named "jsonlines" and is described in the App
An implementation SHOULD always include this format as one of the partial data formats provided for a property that has been omitted from the response to the initial query.
Implementations MAY provide links to their own non-standard formats, but non-standard format names MUST be prefixed by a database-provider-specific prefix.

If the server supports the :query-param:`property_ranges` query parameter, as described in section `Single Entry URL Query Parameters`_, additional metadata SHOULD be present for each field that is returned via the partial data mechanism.
Below follows an example of the :field:`data` and :field:`meta` parts of a response using the JSON response format.
It communicates that the property value has been omitted from the response and includes three different links for different partial data formats provided.

.. code:: jsonc
{
// ...
"data": {
"type": "structures",
"id": "2345678",
"attributes": {
"a": null
}
"meta": {
"partial_data_links": {
"a": [
{
"format": "jsonlines",
"link": "https://example.org/optimade/v1.2/extensions/partial_data/structures/2345678/a/default_format"
},
{
"format": "_exmpl_bzip2_jsonlines",
"link": "https://db.example.org/assets/partial_values/structures/2345678/a/bzip2_format"
},
{
"format": "_exmpl_hdf5",
"link": "https://cloud.example.org/ACCHSORJGIHWOSJZG"
}
]
}
}
}
// ...
}
Metadata for Property_ranges query parameter
--------------------------------------------

If the server supports the :query-param:`property_ranges` query parameter, as described in section `Single Entry URL Query Parameters`_, additional metadata SHOULD be present for each field for which the :query-param:`property_ranges` query parameter can be used.
The client needs this metadata to be able to select only a part of the data of a property.
This metadata is stored in a dictionary in the field :field:`range` which is stored as per property metadata as described in section `Metadata properties`_

- :field:`range`: Dictionary
A dictionary that contains the data necessary for the client to select only a sub_range of a property.
If the property_ranges query parameter is supported for this property the following keys MUST be present.
Otherwise these keys are optional.

- :field:`range_ids`: List of Strings.
A list with an identifier for each dimension of the property.
The outermost dimension of a nested array should come first.

If, within one entry, dimensions for two or more properties share the same :field:`range_id` those dimensions should be thought of as the same dimension.
For example, if both the :property:`energy` and :property:`cartesian_site_positions` of a molecular dynamics trajectory share a range_id of :val:`frames`.
This means that the energy at index x(in the dimension labelled by this range_id) belongs to the cartesian_site_positions at the same index x.
A dictionary that contains the data necessary for the client to select only a sub_range of a property.
If the property_ranges query parameter is supported for this property the following keys MUST be present.
Otherwise this dictionary and these keys are optional.

- :field:`indexable_dim`: List of Strings.
The list of range_ids of the dimensions for which slicing is supported, i.e. the client can request a subrange in this dimension via the :query-param:`property_ranges` query parameter.

- :field:`"layout"`: String.
- :field:`layout`: String.
A string either equal to :val:`"dense"` or :val:`"sparse"` to indicate whether the property is returned in a dense or sparse layout.

- :field:`data_range`: List of Slice Objects.
This field describes how the values are distributed in the different dimensions.
It consists of a slice object for each dimension of the property.
The order of the slice objects must be the same as in the :field:`range_ids` field.
The order of the slice objects must be the same as in the :field:`range_ids` field in the property definition.
If the :field:`layout` field is set to :val:`"sparse"` the value of the :field:`step` sub-field has no meaning.

- :field:`nvalues`: Integer.
The total number of values in the property.
SHOULD be a queryable property with support for all mandatory filter features.

Below follows an example of the :field:`data` and :field:`meta` parts of a response using the JSON response format.
It communicates that the property value has been omitted from the response and includes three different links for different partial data formats provided and metadata that makes it possible to request only a part of the data for this property.
It communicates that the property value has been omitted from the response and metadata that makes it possible to request only a part of the data for this property.

.. code:: jsonc
Expand Down Expand Up @@ -526,20 +558,7 @@ It communicates that the property value has been omitted from the response and i
}
},
"partial_data_links": {
"a": [
{
"format": "jsonlines",
"link": "https://example.org/optimade/v1.2/extensions/partial_data/structures/2345678/a/default_format"
},
{
"format": "_exmpl_bzip2_jsonlines",
"link": "https://db.example.org/assets/partial_values/structures/2345678/a/bzip2_format"
},
{
"format": "_exmpl_hdf5",
"link": "https://cloud.example.org/ACCHSORJGIHWOSJZG"
}
]
//...
}
}
}
Expand Down Expand Up @@ -1221,24 +1240,26 @@ The rules for which properties are to be present for an entry in the response ar
Single Entry URL Query Parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

- **property\_ranges**: A set of slices which specify which range of a property should be returned for partial data properties.
The property to which this range should be applied can be specified via the :query-param:`response_fields` query parameter.
- **property\_ranges**: A set of slices which specify which sub-range of a property should be returned.

The property to which this sub-range should be applied can be specified via the :query-param:`response_fields` query parameter.
Each slice consists of the name of a dimension directly followed by the requested slice in this dimension.
The dimension name and the start, stop and step values of the slice are separated by colons (":", ASCII 58(0x3A))
The slice is defined in the same manner as the `slice object`_.
The slice is defined in a similar manner as the `slice object`_.
If no value is placed between the colons for a component of the slice the default value is used.

The first integer specifies the start of the slice, i.e. the first index in that dimension for which values should be returned.
The default is the index of the first value.
The second integer specifies the end/stop of the slice, i.e. the last index for which values should be returned.
The default is the index of the last value of the property.
And the third integer specifies the step size in that dimension.
The default is :val:`1`

Multiple dimensional slices can be defined by specifying a range for each dimension.
These ranges are separated by a comma (",", ASCII 44(0x2C)).
The slices are 0 based, i.e. the first value has index 0, and inclusive i.e. for the range :val:`:10:20:1` the last value returned belongs to index 20.

In general support is OPTIONAL, property definitions may however deviate from this and place stricter requirements on servers.
Databases MUST use the methods described in the section `Transmission of large property values`_ to return the requested sub-range of a property.
Databases MUST use these ranges for properties where the dimension is listed under indexable_dimensions, if this is not the case the database MAY return more data than was specified in the range.

If a dimension is not specified, it is assumed that the whole range in that dimension is requested.
Expand All @@ -1251,6 +1272,7 @@ Single Entry URL Query Parameters

:query-url:`http://example.com/optimade/v1/structures/id_12345?response_fields=cartesian_site_positions&property_ranges=frames::1000:10,particles:30:70::`

|
The client MAY provide a set of additional URL query parameters for this endpoint type.
URL query parameters not recognized MUST be ignored.
Expand Down Expand Up @@ -2195,6 +2217,19 @@ A Property Definition MUST be composed according to the combination of the requi
- :val:`partial`: the defined property MUST be queryable with support for a subset of the filter language operators as specified by the field :field:`query-support-operators`.
- :val:`none`: the defined property does not need to be queryable with any features of the filter language.

- :field:`property_range`: Dictionary.
A dictionary that contains information needed to use the :query-param:`property_range` query parameter

- :field:`supported`: Boolean.
This field indicates whether the it is possible to use the :query-param:`property_range` query parameter in combination with this field. If true the use of the :query-param:`property_range` is supported

- :field:`range_ids`: List of Strings.
A list with an identifier for each dimension of the property.
The outermost dimension of a nested array should come first.
If, within one entry, dimensions for two or more properties share the same :field:`range_id` those dimensions should be thought of as the same dimension.
For example, if both the :property:`energy` and :property:`cartesian_site_positions` of a molecular dynamics trajectory share a range_id of :val:`frames`.
This means that the energy at index x(in the dimension labelled by this range_id) belongs to the cartesian_site_positions at the same index x.

- :field:`query-support-operators`: List of Strings.
Defines the filter language features supported on this property.
Each string in the list MUST be one of :val:`<`, :val:`<=`, :val:`>`, :val:`>=`, :val:`=`, :val:`!=`, :val:`CONTAINS`, :val:`STARTS WITH`, :val:`ENDS WITH`:, :val:`HAS`, :val:`HAS ALL`, :val:`HAS ANY`, :val:`HAS ONLY`, :val:`IS KNOWN`, :val:`IS UNKNOWN` with the following meanings:
Expand Down

0 comments on commit a84c72e

Please sign in to comment.