Skip to content

Commit

Permalink
update xml generation to support metadata form changes (#111)
Browse files Browse the repository at this point in the history 
* set metadata scope rather then hard coding to dataset

* improve lineage section. not this results in a change to the 'hsitory' attribute which was previously a string and is now expected to be an object.

* fix some xml bugs and update sample reqcord with bilingual values

* adjust template to add related works

* update template to include new lineage and related works fields

* add iso version of scope
  • Loading branch information
@fostermh
fostermh authored Apr 25, 2024
1 parent 1665fef commit 95ae4bb
Show file tree
Hide file tree
Showing 4 changed files with 118 additions and 44 deletions.
96 changes: 65 additions & 31 deletions metadata_xml/iso19115-cioos-template/main.j2
View file
Original file line number Diff line number Diff line change
Expand Up @@ -708,74 +708,108 @@
{% endif %}
{% for history in record['metadata']['history'] %}
<mdb:resourceLineage>
{% for lineageStep in record['metadata']['history'] %}
<mrl:LI_Lineage>
{{- bl.bilingual('mrl:statement', 'statement', history) -}}
{{- bl.bilingual('mrl:statement', 'statement', lineageStep) -}}
{# scope: CIOOS core mandatory #}
<mrl:scope>
<mcc:MD_Scope>
{# level: mandatory #}
<mcc:level>
<mcc:MD_ScopeCode codeList="http://standards.iso.org/iso/19115/resources/Codelists/cat/codelists.xml#MD_ScopeCode" codeListValue="{{- history['scope'] or 'dataset' -}}" />
<mcc:MD_ScopeCode codeList="http://standards.iso.org/iso/19115/resources/Codelists/cat/codelists.xml#MD_ScopeCode" codeListValue="{{lineageStep.scopeIso}}" />
</mcc:level>
</mcc:MD_Scope>
</mrl:scope>
{% for additional_documentation in history['additionalDocumentation'] %}
{% for doc in lineageStep.additionalDocumentation %}
<mrl:additionalDocumentation>
<cit:CI_Citation>
{{- bl.bilingual('cit:title','title', additional_documentation) -}}
{% if additional_documentation['authority'] %}
{{- bl.bilingual('cit:title', 'title', doc) }}
<cit:identifier>
<mcc:MD_Identifier>
<mcc:authority>
<cit:CI_Citation>
<cit:title>
<gco:CharacterString>{{- additional_documentation['authority'] -}}</gco:CharacterString>
</cit:title>
</cit:CI_Citation>
</mcc:authority>
<mcc:code>
<gco:CharacterString>{{- additional_documentation['code'] -}}</gco:CharacterString>
<gco:CharacterString>{{- doc['code'] -}}</gco:CharacterString>
</mcc:code>
</mcc:MD_Identifier>
</cit:identifier>
{% if is_url(doc['code']) %}
<cit:onlineResource>
<cit:CI_OnlineResource>
<cit:linkage>
<gco:CharacterString>{{- doc['code']|e -}}</gco:CharacterString>
</cit:linkage>
<cit:protocol>
<gco:CharacterString>WWW:LINK</gco:CharacterString>
</cit:protocol>
</cit:CI_OnlineResource>
</cit:onlineResource>
{% endif %}
</cit:CI_Citation>
</mrl:additionalDocumentation>
{% endfor %}

{% for source in history['source'] %}
{% for source in lineageStep.source%}
<mrl:source>
<mrl:LI_Source id="lineageSource">
{{- bl.bilingual('mrl:description','description', source) -}}
{% if source['title'] %}
<mrl:LI_Source>
{{- bl.bilingual('mrl:description', 'description', source) }}
<mrl:sourceCitation>
<cit:CI_Citation>
{{- bl.bilingual('cit:title','title', source) -}}
<cit:date gco:nilReason="missing"/>
{% if source['authority'] %}
{{- bl.bilingual('cit:title', 'title', source) }}
<cit:identifier>
<mcc:MD_Identifier>
<mcc:authority>
<cit:CI_Citation>
<cit:title>
<gco:CharacterString>{{- source['authority'] -}}</gco:CharacterString>
</cit:title>
</cit:CI_Citation>
</mcc:authority>
<mcc:code>
<gco:CharacterString>{{- source['code'] -}}</gco:CharacterString>
</mcc:code>
</mcc:MD_Identifier>
</cit:identifier>
{% if is_url(source['code']) %}
<cit:onlineResource>
<cit:CI_OnlineResource>
<cit:linkage>
<gco:CharacterString>{{- source['code']|e -}}</gco:CharacterString>
</cit:linkage>
<cit:protocol>
<gco:CharacterString>WWW:LINK</gco:CharacterString>
</cit:protocol>
</cit:CI_OnlineResource>
</cit:onlineResource>
{% endif %}
</cit:CI_Citation>
</mrl:sourceCitation>
{% endif %}
</mrl:sourceCitation>
</mrl:LI_Source>
</mrl:source>
{% endfor %}

{% for step in lineageStep.processingStep%}
<mrl:processStep>
<mrl:LI_ProcessStep>
{{- bl.bilingual('mrl:description', 'description', step) }}
<mrl:reference>
<cit:CI_Citation>
{{- bl.bilingual('cit:title', 'title', step) }}
<cit:identifier>
<mcc:MD_Identifier>
<mcc:code>
<gco:CharacterString>{{- step['code'] -}}</gco:CharacterString>
</mcc:code>
</mcc:MD_Identifier>
</cit:identifier>
{% if is_url(step['code']) %}
<cit:onlineResource>
<cit:CI_OnlineResource>
<cit:linkage>
<gco:CharacterString>{{- step['code']|e -}}</gco:CharacterString>
</cit:linkage>
<cit:protocol>
<gco:CharacterString>WWW:LINK</gco:CharacterString>
</cit:protocol>
</cit:CI_OnlineResource>
</cit:onlineResource>
{% endif %}
</cit:CI_Citation>
</mrl:reference>
</mrl:LI_ProcessStep>
</mrl:processStep>
{% endfor %}
</mrl:LI_Lineage>
{% endfor %}
</mdb:resourceLineage>
{% endfor %}

Expand Down
4 changes: 4 additions & 0 deletions metadata_xml/template_functions.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,12 @@
from datetime import date
from jinja2 import Environment, FileSystemLoader
from yattag import indent
import validators

SCHEMA_FOLDER_NAME = "iso19115-cioos-template"

def is_url(val):
return val and validators.url(val)

def list_or_value_to_list(val):
"""turns a list or a single value into a list"""
Expand Down Expand Up @@ -83,6 +86,7 @@ def metadata_to_xml(record):
template_env.globals.update(
list_all_languages_in_record=list_all_languages_in_record,
list_or_value_to_list=list_or_value_to_list,
is_url=is_url,
)
template_env.filters["normalize_datestring"] = normalize_datestring
template = template_env.get_template("main.j2")
Expand Down
59 changes: 47 additions & 12 deletions sample_records/record.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,24 +31,56 @@ metadata:

history:
- statement:
en: history in english
fr: history in french
en: English statement
fr: French statement
translations:
fr:
verified: false
message: text translated using the Amazon translate service / texte traduit à l'aide du service de traduction Amazon
scope: model
additionalDocumentation:
- title:
en: title in english
fr: title in french
en: English Title
fr: French Title
translations:
fr:
verified: false
message: text translated using the Amazon translate service / texte traduit à l'aide du service de traduction Amazon
authority: citation authority
code: citation code
source:
- description:
en: description in english
fr: description in french
code: https://doi.org/10.0000/123456
source:
- description:
en: English Description
fr: French Description
translations:
fr:
verified: false
message: text translated using the Amazon translate service / texte traduit à l'aide du service de traduction Amazon
title:
en: title in english
fr: title in french
en: English Title
fr: French Title
translations:
fr:
verified: false
message: text translated using the Amazon translate service / texte traduit à l'aide du service de traduction Amazon
authority: citation authority for source
code: citation code for source
code: https://doi.org/10.0000/123456
processingStep:
- description:
en: English Description
fr: French Description
translations:
fr:
verified: false
message: text translated using the Amazon translate service / texte traduit à l'aide du service de traduction Amazon
title:
en: English Title
fr: French Title
translations:
fr:
verified: false
message: text translated using the Amazon translate service / texte traduit à l'aide du service de traduction Amazon
code: https://doi.org/10.0000/123456

dates:
creation: 2011-11-11
Expand Down Expand Up @@ -100,6 +132,9 @@ identification:
eov:
en: [oxygen]
fr: [oxygène]
taxa:
en: [Animalia, Arthropoda, Insecta, Hymenoptera, Apidae, Apis, Apis mellifera]
fr: [Animalia, Arthropoda, Insecta, Hymenoptera, Apidae, Apis, Apis mellifera]
temporal_begin: 1950-07-31
temporal_end: now
temporal_duration: P1D
Expand Down
3 changes: 2 additions & 1 deletion setup.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@
install_requires=['markupsafe == 2.0.1',
'Jinja2 == 2.10.3',
'PyYAML == 5.1.2',
'yattag == 1.14.0'
'yattag == 1.14.0',
'validators == 0.20.0'
]

)

0 comments on commit 95ae4bb

Please sign in to comment.