diff --git a/optimade.rst b/optimade.rst index b25497623..b91b89d43 100644 --- a/optimade.rst +++ b/optimade.rst @@ -456,7 +456,7 @@ How this list is provided is response format-dependent. For the JSON response format, see the description of the :field:`partial_data_links` field, nested under :field:`data` and then :field:`meta`, in the section `JSON Response Schema: Common Fields`_. The default partial data format is named "jsonlines" and is described in the Appendix `OPTIMADE JSON lines partial data format`_. -An implementation SHOULD always include this format as one of alternative partial data formats provided for a property that has been omitted from the response to the initial query. +An implementation SHOULD always include this format as one of the partial data formats provided for a property that has been omitted from the response to the initial query. Implementations MAY provide links to their own non-standard formats, but non-standard format names MUST be prefixed by a database-provider-specific prefix. Below follows an example of the :field:`data` and :field:`meta` parts of a response using the JSON response format that communicates that the property value has been omitted from the response, with three different links for different partial data formats provided. @@ -506,7 +506,7 @@ For example, the metadata property definition of the field :field:`_exmpl_exampl The reason for this limitation is to avoid name collisions with metadata fields defined by the OPTIMADE standard in the future that apply also to database-specific data fields. Implementation of the :field:`meta` field is OPTIONAL. -However, when an implementation supports the :field:`property_metadata` field, it SHOULD include metadata fields for all fields present in the data part of the response which has metadata. +However, when an implementation supports the :field:`property_metadata` field, it SHOULD include metadata fields for all properties which have metadata and are present in the data part of the response. Example of a response in the JSON response format with two structure entries that each include a metadata property for the attribute field :field:`element_ratios` and the database-specific per entry metadata field :field:`_exmpl_originates_from_project` : @@ -751,11 +751,11 @@ Every response SHOULD contain the following fields, and MUST contain at least :f - **partial_data_links**: an object used to list links which can be used to fetch data that has been omitted from the :field:`data` part of the response. The keys are the names of the fields in :field:`attributes` for which partial data links are available. - Each value is a list of items that MUST have the following keys: + Each value is a list of objects that MUST have the following keys: - **format**: String. - A name of the format provided via this link. - One of the items SHOULD be "jsonlines", which refers to the format in `OPTIMADE JSON lines partial data format`_. + The name of the format provided via this link. + For one of the objects this :field:`format` field SHOULD have the value "jsonlines", which refers to the format in `OPTIMADE JSON lines partial data format`_. - **link**: String. A `JSON API link `__ that points to a location from which the omitted data can be fetched. @@ -3667,7 +3667,7 @@ The markers have been deliberately designed to be valid JSON objects but *not* v Since the OPTIMADE list data type is defined as a list of values of the same data type or :val:`null`, the above markers cannot be encountered inside the actual data of an OPTIMADE property. **Implementation note:** the recognizable string values for the markers should make it possible to prescreen the raw text of the JSON data lines for the reference-marker string to determine which are the lines that one can exclude from further processing to resolve references (alternatively, this screening can be done by the string parser used by the JSON parser). - The undelying design idea is that for lines that have reference-markers, the time it takes to process the data structure to locate the markers should be negligible compared to the time it takes to resolve and handle the large data they reference. + The underlying design idea is that for lines that have reference-markers, the time it takes to process the data structure to locate the markers should be negligible compared to the time it takes to resolve and handle the large data they reference. Hence, the most relevant optimization is to avoid spending time processing data structures to find markers for lines where there are none. The full response MUST be valid `JSON Lines `__ that adheres to the following format: @@ -3685,8 +3685,10 @@ The header object MUST contain the keys: It MUST contain the following key: - :field:`"format"`: String. - Specifies the minor version of the partial data format used. The string MUST be of the format "MAJOR.MINOR", referring to the version of the OPTIMADE standard that describes the format. The version number string MUST NOT be prefixed by, e.g., "v". In implementations of the present version of the standard, the value MUST be exactly :val:`1.2`. - A client MUST NOT expect to be able to parse the format if the field is not a string of the format MAJOR.MINOR or if the MAJOR version number is unrecognized. + Specifies the minor version of the partial data format used. + The string MUST be of the format "MAJOR.MINOR", referring to the version of the OPTIMADE standard that describes the format. + The version number string MUST NOT be prefixed by, e.g., "v". In implementations of the present version of the standard, the value MUST be exactly :val:`1.2`. + A client MUST NOT expect to be able to parse the :field:`format` value if the field is not a string of the format MAJOR.MINOR or if the MAJOR version number is unrecognized. - :field:`"layout"`: String. A string either equal to :val:`"dense"` or :val:`"sparse"` to indicate whether the returned format uses a dense or sparse layout. @@ -3695,7 +3697,7 @@ The following key is RECOMMENDED in the header object: - :field:`"returned_ranges"`: Array of Object. For dense layout, and sparse layout of one dimensional list properties, the array contains a single element which is a `slice object`_ representing the range of data present in the response. - In the specific case of a hierarchy of list properties represented as a sparse multi-dimensional array, if the field :field:`"returned_ranges"` is given, it MUST contain one slice object per dimension of the multi-dimensional array, representing slices for each dimension that cover the data given in the response. + In the specific case of a hierarchy of list properties represented as a sparse multidimensional array, if the field :field:`"returned_ranges"` is given, it MUST contain one slice object per dimension of the multidimensional array, representing slices for each dimension that cover the data given in the response. The header object MAY also contain the keys: @@ -3732,18 +3734,18 @@ The header object MAY also contain the keys: The format and requirements on this schema are the same as for the inline schema field :field:`item_schema`. The format of data lines of the response (i.e., all lines except the first and the last) depends on whether the header object specifies the layout as :val:`"dense"` or :val:`"sparse"`. -- **Dense layout:** In the dense partial data layout, each data line reproduces one list item in the OPTIMADE list property being transmitted in JSON format. +- **Dense layout:** In the dense partial data layout, each data line reproduces one item from the OPTIMADE list property being transmitted in the JSON format. If OPTIMADE list properties are embedded inside the item, they can either be included in full or replaced with a reference-marker. If a list is replaced by a reference marker, the client MAY use the provided URL to obtain the list items. If the field :field:`"returned_ranges"` is omitted, then the client MUST assume that the data is a continuous range of data from the start of the array up to the number of elements given until reaching the end-of-data-marker or next-marker. -- **Sparse layout for one-dimensional list:** When the response sparsely communicates items for a one-dimensional OPTIMADE list property, each data line contains a JSON array on the format: +- **Sparse layout for one-dimensional list:** When the response sparsely communicates items for a one-dimensional OPTIMADE list property, each data line contains a JSON array of the format: - The first item of the array is the zero-based index of list property item being provided by this line. - The second item of the array is the list property item located at the indicated index, represented using the same format as each line in the dense layout. In the same way as for the dense layout, reference-markers are allowed inside the item data for embedded lists that do not fit in the response (see example below). -- **Sparse layout for multi-dimensional lists:** the server MAY use a specific sparse layout for the case that the OPTIMADE property represents a series of directly hierarchically embedded lists (i.e., a multidimensional sparse array). +- **Sparse layout for multidimensional lists:** the server MAY use a specific sparse layout for the case that the OPTIMADE property represents a series of directly hierarchically embedded lists (i.e., a multidimensional sparse array). In this case, each data line contains a JSON array of the format: - All array items except the last one are integer zero-based indices of the list property item being provided by this line; these indices refer to the aggregated dimensions in the order of outermost to innermost. @@ -3785,7 +3787,7 @@ The third provided item (index 14 in the original list) is only partially return [[11, 110], ["PARTIAL-DATA-REF", ["https://example.db.org/value3"]], [550, 333]] ["PARTIAL-DATA-NEXT", ["https://example.db.org/value4"]] -Below follows an example of the sparse layout for multi-dimensional lists with three aggregated dimensions. +Below follows an example of the sparse layout for multidimensional lists with three aggregated dimensions. The underlying property value can be taken to be sparse data in lists in four dimensions of 10000 x 10000 x 10000 x N, where the innermost list is a non-sparse list of arbitrary length of numbers. The only non-null items in the outer three dimensions are, say, [3,5,19], [30,15,9], and [42,54,17]. The response below communicates the first item explicitly; the second one by deferring the innermost list using a reference-marker; and the third item is not included in this response, but deferred to another page via a next-marker. @@ -3797,7 +3799,7 @@ The response below communicates the first item explicitly; the second one by def [30,15,9, ["PARTIAL-DATA-REF", ["https://example.db.org/value1"]]] ["PARTIAL-DATA-NEXT", ["https://example.db.org/"]] -An example of the sparse layout for multi-dimensional lists with three aggregated dimensions and integer values: +An example of the sparse layout for multidimensional lists with three aggregated dimensions and integer values: .. code:: json @@ -3806,7 +3808,7 @@ An example of the sparse layout for multi-dimensional lists with three aggregate [30,15,9, 31] ["PARTIAL-DATA-NEXT", ["https://example.db.org/"]] -An example of the sparse layout for multi-dimensional lists with three aggregated dimensions and values that are multidimensional lists of integers of arbitrary lengths: +An example of the sparse layout for multidimensional lists with three aggregated dimensions and values that are multidimensional lists of integers of arbitrary lengths: .. code:: json