diff --git a/_docs/gold-standard-examples.md b/_docs/gold-standard-examples.md index 1a7ef10..fa713c4 100644 --- a/_docs/gold-standard-examples.md +++ b/_docs/gold-standard-examples.md @@ -10,7 +10,7 @@ summary: Describes gold standard examples for various dataset types and scenari ## **Overview** -The page provides links to example datasets that satisfy the [IOOS Metadata Profile 1.2](ioos-metadata-profile-v1-2.html). +The page provides links to example datasets that satisfy the [IOOS Metadata Profile 1.2](./ioos-metadata-profile-v1-2). Since the IOOS standard builds off of the [**NOAA NCEI NetCDF Templates**](https://www.nodc.noaa.gov/data/formats/netcdf/), the NCEI "Gold Standard" examples ([HTTP](https://data.nodc.noaa.gov/ncei/example/data/netcdf/), [THREDDS](https://data.nodc.noaa.gov/thredds/catalog/example/catalog.html)) provide a great starting point. diff --git a/_docs/index.md b/_docs/index.md index 82e6f07..3789e4d 100644 --- a/_docs/index.md +++ b/_docs/index.md @@ -9,7 +9,7 @@ summary: The IOOS Metadata Profile contains dataset attribution guidelines and e ## **Current Profile Version** -### [**IOOS Metadata Profile 1.2**](ioos-metadata-profile-v1-2.html) +### [**IOOS Metadata Profile 1.2**](ioos-metadata-profile-v1-2) Looking for the latest version? Follow the link above for details on requirements to implement the IOOS Metadata Profile for publishing data according to the latest IOOS guidelines. @@ -29,7 +29,7 @@ Individual releases of the IOOS Metadata Profile (e.g. 1.0, 1.1, 1.2) target spe ## Profile Versions: -### [**IOOS Metadata Profile, Version 1.2** (Current)](ioos-metadata-profile-v1-2.html) +### [**IOOS Metadata Profile, Version 1.2** (Current)](ioos-metadata-profile-v1-2) IOOS Metadata Profile Version 1.2 (released in 2020) is the current **valid**{: style="color: green"} profile, and is based upon the following convention versions: @@ -58,7 +58,7 @@ Version 1.2 of the Metadata Profile targets ERDDAP as the future platform for IO Data providers wishing to maintain ncSOS/IOOS SOS compatibility can refer to this list and add all required IOOS attributes from deprecated Metadata Profile 1.1 and or duplicate attributes where appropriate. -### [**IOOS Metadata Profile, Version 1.1**](ioos-metadata-profile-v1-1.html) +### [**IOOS Metadata Profile, Version 1.1**](ioos-metadata-profile-v1-1) IOOS Metadata Profile Version 1.1 (**deprecated in 2019**{: style="color: red"}) is based upon the following convention versions: @@ -66,7 +66,7 @@ IOOS Metadata Profile Version 1.1 (**deprecated in 2019**{: style="color: red"}) - [Attribute Convention for Data Discovery 1.3](http://wiki.esipfed.org/index.php/Attribute_Convention_for_Data_Discovery_1-3) - [NOAA NCEI NetCDF Templates 2.0](https://www.nodc.noaa.gov/data/formats/netcdf/v2.0/) -### [**IOOS Metadata Profile, Version 1.0**](ioos-metadata-profile-v1-0.html) +### [**IOOS Metadata Profile, Version 1.0**](ioos-metadata-profile-v1-0) IOOS Metadata Profile Version 1.0 (**deprecated in 2016**{: style="color: red"}) is based upon the following convention versions: diff --git a/_docs/ioos-metadata-profile-v1-0.md b/_docs/ioos-metadata-profile-v1-0.md index b807aec..71ceb40 100644 --- a/_docs/ioos-metadata-profile-v1-0.md +++ b/_docs/ioos-metadata-profile-v1-0.md @@ -4,7 +4,7 @@ keywords: [ioos, metadata, netCDF, 1.0] tags: [ioos, metadata, netCDF, 1.0] toc: false #permalink: index.html -summary: Deprecated 2016. Visit the [1.2 Metadata Profile](./ioos-metadata-profile-v1-2.html) page for the current version. +summary: Deprecated 2016. Visit the [1.2 Metadata Profile](./ioos-metadata-profile-v1-2) page for the current version. --- @@ -18,7 +18,7 @@ summary: Deprecated 2016. Visit the [1.2 Metadata Profile](./ioos-metadata-prof 2. Each attribute in the Profile is either **required** or **recommended**. 3. All **required** attributes must have meaningful values assigned to them in accordance with the rules prescribed by the corresponding Convention or Template. 4. Each and all of the **recommended** attributes may be omitted; however, it is highly desirable that these attributes are included into the NetcDF metadata ***AND*** have meaningful values assigned to them. - 5. The **`platform_variable:ioos_code`** and **`platform_variable:short_name`** are the only **interchangeable** attributes - either a single **`platform_variable:ioos_code`** or a combination of **`platform_variable:short_name`** with **`naming_authority`** is **required** to ensure that ncSOS will be able to produce the IOOS SOS Asset Identifier for the specific platform (see the [NetCDF to IOOS SOS Crosswalk](https://github.com/ioos/ioos-metadata/blob/master/docs/NetCDF-to-SOS%20Mappings_clean_2016-04-07a.xlsx) for details). The rest of attributes ***may not*** be substituted for one another. + 5. The **`platform_variable:ioos_code`** and **`platform_variable:short_name`** are the only **interchangeable** attributes - either a single **`platform_variable:ioos_code`** or a combination of **`platform_variable:short_name`** with **`naming_authority`** is **required** to ensure that ncSOS will be able to produce the IOOS SOS Asset Identifier for the specific platform (see the [NetCDF to IOOS SOS Crosswalk](https://github.com/ioos/ioos-metadata/raw/gh-pages/_docs/NetCDF-to-SOS%20Mappings_clean_2016-04-07a.xlsx) for details). The rest of attributes ***may not*** be substituted for one another. 6. The **`platform_vocabulary`** attribute is at the moment the only pure ACDD v1.3 attribute that is included in the Profile. 7. This document only describes a fraction of the Profile: - attributes that are IOOS-specific; @@ -46,7 +46,7 @@ creator_url | ACDD | The URL of the institution that collected the data. | glob creator_zipcode | IOOS | ZIP code of the person or organization that collected the data. | global | recommended featureType | CF | CF attribute for identifying the featureType, e.g. featureType = "timeSeries". | global | required geophysical_variable:_FillValue
geospatial_variable:_FillValue | CF | This value is considered to be a special value that indicates undefined or missing data, and is returned when reading values that were not written: {::nomarkdown}{:/} | variable | recommended -geophysical_variable:standard_name | CF | Standardized field which uses the [CF Standard Names](http://www.cfconventions.org/documents.html/). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | variable | required +geophysical_variable:standard_name | CF | Standardized field which uses the [CF Standard Names](https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html/). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | variable | required id | ACDD | An identifier for the data set, provided by and unique within its naming authority. The combination of the **`naming authority`** and the **`id`** should be globally unique, but the **`id`** can be globally unique by itself also. IDs can be URLs, URNs, DOIs, meaningful text strings, a local key, or any other unique string of characters. The **`id`** should not include blanks. | global | required institution | ACDD | The institution of the person or group that collected the data. | global | required instrument_variable:discriminant | IOOS | The value of a **`discriminant`** applies to the like-named field in the IOOS SOS Asset Identifier URN; it ensures that in case of multiple sensors measuring the same **`observedProperty`**, each sensor has a unique ID.

Examples: {::nomarkdown}{:/}| variable | required, if applicable @@ -68,7 +68,7 @@ publisher_phone | IOOS | The phone number of the person or group that distribute publisher_state | IOOS | State of the person or organization that distributes the data. | global | recommended publisher_url | ACDD | URL of the person or group that distributes the data files. | global | recommended publisher_zipcode | IOOS | ZIP code of the person or organization that distributes the data. | global | recommended -standard_name_vocabulary | ACDD | Standardized field which uses the [CF Standard Names](http://www.cfconventions.org/documents.html/). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | global | required +standard_name_vocabulary | ACDD | Standardized field which uses the [CF Standard Names](https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html/). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | global | required summary | ACDD | One paragraph describing the data set. | global | recommended title | ACDD | One sentence about the data contained within the file. | global | required units | CF | Required for most all variables that represent dimensional quantities. The value should come from [**`udunits`**](http://www.unidata.ucar.edu/software/udunits/) authoritative vocabulary, which is documented in the CF standard name table with it's corresponding standard name. The **`udunits`** package includes a file `udunits.dat` which lists its supported unit names. | variable | required diff --git a/_docs/ioos-metadata-profile-v1-1.md b/_docs/ioos-metadata-profile-v1-1.md index e874870..c0adfbc 100644 --- a/_docs/ioos-metadata-profile-v1-1.md +++ b/_docs/ioos-metadata-profile-v1-1.md @@ -4,7 +4,7 @@ keywords: [ioos, metadata, netCDF, 1.1] tags: [ioos, metadata, netCDF, 1.1] toc: false #permalink: index.html -summary: Deprecated 2019. Visit the [1.2 Metadata Profile](./ioos-metadata-profile-v1-2.html) page for the current version. +summary: Deprecated 2019. Visit the [1.2 Metadata Profile](./ioos-metadata-profile-v1-2) page for the current version. --- ## **Revision History** @@ -12,7 +12,7 @@ summary: Deprecated 2019. Visit the [1.2 Metadata Profile](./ioos-metadata-pro | Version | Description | Date | |:--- |:--- |:--- | -| 1.0 | [Initial version based on the NODC Templates 1.1 and ACDD 1.1](./ioos-metadata-profile-v1-0.html) | 2016-10-01 | +| 1.0 | [Initial version based on the NODC Templates 1.1 and ACDD 1.1](./ioos-metadata-profile-v1-0) | 2016-10-01 | | 1.1 | Updated version based on the NCEI Templates 2.0 and ACDD 1.3 | 2016-11-01 | @@ -51,7 +51,7 @@ creator_url | ACDD | The URL of the institution that collected the data. | glob creator_zipcode | IOOS | ZIP code of the person or organization that collected the data. | global | recommended featureType | CF | CF attribute for identifying the featureType, e.g. featureType = "timeSeries". | global | required geophysical_variable:_FillValue
geospatial_variable:_FillValue | CF | This value is considered to be a special value that indicates undefined or missing data, and is returned when reading values that were not written: {::nomarkdown}{:/} | variable | recommended -geophysical_variable:standard_name | CF | Standardized field which uses the [CF Standard Names](http://www.cfconventions.org/documents.html/). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | variable | required +geophysical_variable:standard_name | CF | Standardized field which uses the [CF Standard Names](https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | variable | required id | ACDD | An identifier for the data set, provided by and unique within its naming authority. The combination of the **`naming authority`** and the **`id`** should be globally unique, but the **`id`** can be globally unique by itself also. IDs can be URLs, URNs, DOIs, meaningful text strings, a local key, or any other unique string of characters. The **`id`** should not include blanks. | global | required institution | ACDD | The institution of the person or group that collected the data. | global | required instrument_variable:discriminant | IOOS | The value of a **`discriminant`** applies to the like-named field in the IOOS SOS Asset Identifier URN; it ensures that in case of multiple sensors measuring the same **`observedProperty`**, each sensor has a unique ID.

Examples: {::nomarkdown}{:/}| variable | required, if applicable @@ -62,7 +62,7 @@ platform | ACDD | Name of the platform(s) that supported the sensor data used to platform_variable:ioos_code | IOOS | Provides IOOS asset identification similar to **`wmo_code`** and **`nodc_code`**. The attribute is a URN that should follow the "[IOOS Convention for Asset Identification](http://ioos.github.io/conventions-for-observing-asset-identifiers/ioos-assets-v1-0.html)" with a general pattern of _**`urn:ioos:asset_type:authority:label[:discriminant]`**_.

Examples: {::nomarkdown}{:/}
**NOTE:** interchangeable with **`platform_variable:short_name`** | variable | required platform_variable:long_name | NCEI Templates | Provide a descriptive, long name for this variable. | variable | required platform_variable:short_name | IOOS | Provide a short name for the platform. Similar to ID, a **`short_name`** can be any unique string of characters that does not include blanks.

Examples: {::nomarkdown}{:/}
**NOTE:** interchangeable with **`platform_variable:ioos_code`** | variable | required -platform_variable:type | IOOS | In conjunction with a **`platform_vocabulary`** attribute, identifies platform's type as defined in the [IOOS Platform Categories vocabulary](https://mmisw.org/orr/#http://mmisw.org/ont/ioos/platform), or [SeaVoX Platform Categories vocabulary](http://vocab.nerc.ac.uk/collection/L06/current/"), or any other vocabulary. The URL of the actual vocabulary must be published in the **`platform_vocabulary`** global attribute.

Alternatively, the **`platform`** and **`platform_vocabulary`** pair of attributes may be used; however, this option is not recommended (see details in the **`platform_vocabulary`** description.) | variable | required +platform_variable:type | IOOS | In conjunction with a **`platform_vocabulary`** attribute, identifies platform's type as defined in the [IOOS Platform Categories vocabulary](https://mmisw.org/orr/#http://mmisw.org/ont/ioos/platform), or [SeaVoX Platform Categories vocabulary](http://vocab.nerc.ac.uk/collection/L06/current/), or any other vocabulary. The URL of the actual vocabulary must be published in the **`platform_vocabulary`** global attribute.

Alternatively, the **`platform`** and **`platform_vocabulary`** pair of attributes may be used; however, this option is not recommended (see details in the **`platform_vocabulary`** description.) | variable | required platform_vocabulary | ACDD | Controlled vocabulary for the names used in the "platform" attribute.

It is recommended that this attribute is used in conjunction with the **`platform_variable:type`** attribute. In that case, the recommended value for the **`platform_vocabulary`** attribute is a URL to either the [IOOS Platform Category vocabulary](https://mmisw.org/orr/#http://mmisw.org/ont/ioos/platform), or [SeaVoX Platform Categories vocabulary](http://vocab.nerc.ac.uk/collection/L06/current/).

Example:
**`platform_vocabulary = "https://mmisw.org/orr/#http://mmisw.org/ont/ioos/platform"`**

As an alternative (although not recommended), a NetCDF file may follow the NCEI Template v2.0, which suggests the use of "NASA GCMD Platform Keywords Version 8.1" string as the fixed value for the **`platform_vocabulary`**, and does not stipulate for the **`platform_variable:type`**. Instead, the actual type of the platform must be placed in the global **`platform`** attribute as described in the Science Keyword Rules (http://gcmd.nasa.gov/learn/rules.html) for NASA Global Change Master Directory (GCMD) Keywords (http://gcmd.nasa.gov/learn/keywords.html).

Example:
**`platform: In Situ Ocean-based Platforms > MOORINGS`** | global | required publisher_address | IOOS | Street address of the person or organization that distributes the data. | global | recommended publisher_city | IOOS | City of the person or organization that distributes the data. | global | recommended @@ -73,7 +73,7 @@ publisher_phone | IOOS | The phone number of the person or group that distribute publisher_state | IOOS | State of the person or organization that distributes the data. | global | recommended publisher_url | ACDD | URL of the person or group that distributes the data files. | global | recommended publisher_zipcode | IOOS | ZIP code of the person or organization that distributes the data. | global | recommended -standard_name_vocabulary | ACDD | Standardized field which uses the [CF Standard Names](http://www.cfconventions.org/documents.html/). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | global | required +standard_name_vocabulary | ACDD | Standardized field which uses the [CF Standard Names](https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html/). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | global | required summary | ACDD | One paragraph describing the data set. | global | recommended title | ACDD | One sentence about the data contained within the file. | global | required units | CF | Required for most all variables that represent dimensional quantities. The value should come from [**`udunits`**](http://www.unidata.ucar.edu/software/udunits/) authoritative vocabulary, which is documented in the CF standard name table with it's corresponding standard name. The **`udunits`** package includes a file `udunits.dat` which lists its supported unit names. | variable | required diff --git a/_docs/ioos-metadata-profile-v1-2.md b/_docs/ioos-metadata-profile-v1-2.md index f8f18cc..38c366f 100644 --- a/_docs/ioos-metadata-profile-v1-2.md +++ b/_docs/ioos-metadata-profile-v1-2.md @@ -12,8 +12,8 @@ summary: This is the currently active IOOS Metadata Profile version. See links | Version | Description | Date | |:--- |:--- |:--- | -| 1.0 | [Initial version based on the NODC Templates 1.1 and ACDD 1.1](./ioos-metadata-profile-v1-0.html) | 2016-10-01 | -| 1.1 | [Updated version based on the NCEI Templates 2.0 and ACDD 1.3](./ioos-metadata-profile-v1-1.html) | 2016-11-01 | +| 1.0 | [Initial version based on the NODC Templates 1.1 and ACDD 1.1](./ioos-metadata-profile-v1-0) | 2016-10-01 | +| 1.1 | [Updated version based on the NCEI Templates 2.0 and ACDD 1.3](./ioos-metadata-profile-v1-1) | 2016-11-01 | | **1.2** |**Currently Active Version**
Updated to reflect new IOOS attribution guidance and ERDDAP implementation:
* Add `infoUrl` and `Conventions`
* Make `creator_institution`, `creator_url`, `license`, `publisher_url`, and `summary` required
* Add `contributor_url`, `contributor_email`, and `contributor_role_vocabulary` (recommended)
* Make `contributor_name`, `contributor_role`, `institution`, and `publisher_name` recommended (previously were required)
* Clarify default vocabulary for `contributor_role` and `contributor_role_vocabulary`
* Clarify use of `contributor_name` and `contributor_role` for multiple contributors
* Restrict the profile to allow only a single Platform per dataset; clarify use of 'Platform' variable and related `platform` global and variable attributes
* Add global `platform_id`, `platform_name`, and `wmo_platform_code`
* Remove `platform_variable:ioos_code`, `platform_variable:short_name`, `platform_variable:long_name` and `platform_variable:type`
* Change `creator_zipcode` and `publisher_zipcode` to `creator_postalcode` and `publisher_postalcode`
* Add `geophysical_variable:standard_name_url`, `geophysical_variable:accuracy`, `geophysical_variable:precision`, `geophysical_variable:resolution`
* Add `instrument_vocabulary`
* Add `instrument_variable:calibration_date`, `instrument_variable:component`, `instrument_variable:make_model`
* Add `gts_ingest` to indicate datasets and variables intended for NDBC/GTS harvest
* Add `ioos_ingest` to indicate datasets intended to be harvested into IOOS national products
* Add [Quality Control/QARTOD](#quality-controlqartod) section describing QARTOD flag variable requirements
* Add [Requirements for NDBC/GTS Ingest](#requirements-for-ioos-dataset-ndbcgts-ingest) section
* Add `creator_institution_url` suggested attribute | **2020-01-10** | ## Notes/Caveats @@ -38,7 +38,7 @@ summary: This is the currently active IOOS Metadata Profile version. See links 1. For in situ observation datasets, the IOOS Profile allows only one **'platform'** per dataset. Please see the corresponding [Platform](#platform) section of the profile below for more information. -1. The [**U.S. IOOS National Glider Data Assembly Center**](https://gliders.ioos.us/index.html) currently uses a slightly different [netCDF File Format (V2)](https://ioos.github.io/ioosngdac/ngdac-netcdf-file-format-version-2); work is in progress to harmonize the NGDAC File Format and IOOS Metadata Profile. +1. The [**U.S. IOOS National Glider Data Assembly Center**](https://gliders.ioos.us/index.html) currently uses a slightly different [netCDF File Format (V2)](https://ioos.github.io/glider-dac/ngdac-netcdf-file-format-version-2.html); work is in progress to harmonize the NGDAC File Format and IOOS Metadata Profile. ## Gold Standard Example Datasets @@ -181,7 +181,7 @@ geophysical_variable:accuracy | IOOS | The sensor accuracy is the closeness of t geophysical_variable:missing_value | CF | This should always be equal to the `_FillValue` attribute and both are used for legacy library support. {::nomarkdown}{:/} | variable | recommended geophysical_variable:precision | IOOS | The sensor precision is the closeness of the measurements to each other. It should be given in the same units as the measured variable. If the instrument has been calibrated multiple times with different results, the most recent precision should be provided here (see **`instrument_variable:calibration_date`**). | variable | recommended geophysical_variable:resolution | IOOS | The sensor resolution is the smallest change it can represent in the quantity that it is measuring. It should be given in the same units as the measured variable. | variable | recommended -geophysical_variable:standard_name | CF | Standardized field which uses the [CF Standard Names](http://www.cfconventions.org/documents.html). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | variable | **required** +geophysical_variable:standard_name | CF | Standardized field which uses the [CF Standard Names](https://cfconventions.org/Data/cf-standard-names/current/build/cf-standard-name-table.html). If a variables does not have an existing standard_name in the CF-managed list, this attribute should not be used. In these cases, a standard name can be proposed to the CF community for consideration and acceptance. | variable | **required** geophysical_variable:standard_name_url | IOOS | The URL of a **`standard_name`** in the online vocabulary listed in the global **`standard_name_vocabulary`** attribute.

Example: {::nomarkdown}{:/} | variable | recommended geophysical_variable:units | CF | Required for most all variables that represent dimensional quantities. The value for a geophysical variable's **`units`** attribute should match or be derived from the canonical units specified for the variable's **`standard_name`** in the CF Standard Name table.

CF units are specified by the [**`udunits`**](https://www.unidata.ucar.edu/software/udunits/) package, which includes a file `udunits.dat` listing the valid individual unit names (e.g., "g" and "m") from which which composite **`units`** strings can be formed (e.g., "kg m-3").

For example, all temperature standard names have canonical units of "K", but often geophysical variables that measure temperature are specified with **`units`** of `degree_Celsius` or some variant thereof. | variable | **required** @@ -329,7 +329,7 @@ gts_ingest | IOOS | **Variable** attribute, used in concert with the global equ #### Example -Taken from [PacIOOS's AWS-HIMB ERDDAP Dataset](https://pae-paha.pacioos.hawaii.edu/erddap/info/AWS-HIMB/index.html). +Taken from [PacIOOS's AWS-HIMB ERDDAP Dataset](https://pae-paha.pacioos.hawaii.edu/erddap/info/aws_himb/index.html). ``` NC_GLOBAL {