Merge pull request #78 from OP-TED/release/1.0.0

Release/1.0.0

CHANGELOG.md

@@ -1,19 +1,13 @@
-# SDK 0.7.0 Release Notes
+# SDK 1.0.0 Release Notes
 
 Below is a list of the major updates made to the SDK in this release.
 
 The documentation for the SDK is available at https://docs.ted.europa.eu. The source for this documentation is maintained in the [eforms-docs](https://github.com/OP-TED/eforms-docs) repository.
 
-## Notice types
-A new file format for the notice type definitions is being introduced. The reason for this breaking change is that the notice type definitions included in previous versions of the SDK, were hand-written before the SDK itself was introduced, in order to cover the immediate needs of our notice filling application (eNotices2). The new file format is quite similar to the old one, but simpler to consume and understand. Most importantly the new notice type definition files are generated from our central eForms Metadata Repository which guaranties that there are no conflicting or unnecessary metadata included in them.
-
-:construction: _This is a breaking change. If your application uses the notice type definitions you will need to update it to read the new file format._
-
-
 ## Codelists
-The notice subtypes indicated in the  the tailored codelist `qu-sy` have been corrected.
+The `codelists` folder now contains an index file named `codelists.json`, which describes each available codelist.
 
-A new tailored codelist named `social-service-cpv` has been added.
+A new tailored codelist named `lawful-country` has been added.
 
 
 ## eForms schemas
@@ -22,24 +16,45 @@ No changes in this version.
 
 ## eForms expression language
 Various new features have been added to the eForms expression language (EFX), with a syntax inspired by XPath:
-* conditional expressions: `if ... then ... else ...`
-* for expressions: `for ... in ... return ...`
-* quantified expressions: `every/some ... in ... satisfies ...`
-* sum and count functions now work with sequences 
+* expression to check that the value of a field appears only once : `BT-137-Lot is unique in /BT-137-Lot`
+* function to eliminate duplicates in a sequence: `distinct-values`
+* function to check that 2 sequences contain the same values: `sequence-equal`
+* functions to combine sequences: `value-union`, `value-intersect`, `value-except`
+* for and quantified expressions now allow several variable declarations
+* variables can now be used as context: `$variable::BT-137-Lot`
+
+The syntax for context declaration for expressions is now aligned with the syntax used in templates:
+`{ND-LotTenderingProcess} ${BT-115-Lot == FALSE}`
+
+EFX expressions can now accept parameters: `{ND-Root, number:$p1} ${$p1 == 1}`
+The value of parameters must be provided when the expression is translated.
 
-In addition, the format for dates and times now allows the indication of a time zone offset.
+In EFX templates, you can now add custom numbering at any indentation level.
 
 You can find more information about EFX in the [corresponding section of the documentation](https://docs.ted.europa.eu/eforms/0.7.0/efx).
 
 :construction: _This is a breaking change. Your EFX translator will need to implement a few additional methods to support the new language features._
 
+
 ## Schematron rules
 
-### Co-constraint rules
-We have added a new Schematron file named `stage-5.sch`, containing rules that check the conformity of the values in a notice.
+### Folder structure and configuration file
+Schematron rules are now organised in 2 folders:
+* `static`: all rules that only use information in the notice being validated
+* `dynamic`: all rules in `static`, plus rules that use information in other notices.
+
+The files were renamed to be more consistent: `entry.sch` is now `complete-validation.sch`, and all stages are named `validation-stage-*.sch`
+
+Some rules in `dynamic` retrieve another notice based on its identifier, by making an HTTP request to an external service.
+The URL of this service is configured by a variable defined in the new file `config.sch`.
+
+The file `stage-4-custom.sch` was removed, all the rules it contained are now covered in `validation-stage-5.sch`.
+
+:construction: _This is a breaking change. If you use our Schematron rules you need to choose one of the 2 folders depending on your situation._
 
-### Additional information in validation reports
-For certain types of rules, failures in the validation report did not indicate the exact element that caused the problem, for example if a mandatory element is missing. We have added additional information in the schematron rules, using the `diagnostics` element and attribute, so that those failures now give the XPath of the element under `diagnostic-reference`.
+### New type of "warning" rules
+We have added several rules to try to check if there is a legal basis for publication of a notice in the Supplement to the Official Journal of the European Union. Those rules are indicated as a warning, and recognizable via a `flag="LAWFULNESS"` attribute.
+If one or more of those rules cause a `failed-assert` in the validation report, it means that we need to manually review the notice before it can be published.
 
 ### Added and updated rules
 Various new rules have been added, in particular for information that is allowed or mandatory only under certain conditions.
@@ -49,34 +64,64 @@ Existing rules have been refined and improved.
 ## Example Notices
 The XML notices in the `examples` folder have been updated to take into account the updated schematron rules.
 
-The validation reports have been regenerated, incorporating all the changes listed above.
+The validation reports have been regenerated, using the rules in the `dynamic` folder, incorporating all the changes listed above.
 
 
 ## Fields
 
-### New properties
-We've added new field properties named `idScheme` and `idSchemes` that provide the necessary information for creating (as well as referencing) identifiers of repeatable entities. This information was previously encoded only in the notice type definitions and needed to be normalized.
+### "Unpublished" fields
+Fields that can be withheld from publication ("unpublished") now have a `privacy` property that gives the information needed to withhold publication.
 
-Some repeatable nodes in the XML structure now have an additional property named `identifierFieldId` which points to the field where the identifiers of their instances should be stored.
+### Assertion
+A new property named `assert` gives an assertion, as a boolean EFX expression, that is expected to evaluate to "true".
+This is currently used to specify rules on the field value that can depend on other fields, for example "field A is greated that field B".
+
+### Information for change notices
+In order to specify what information cannot be modified in a change notice compared to the original notice, we have added a new property named `inChangeNotice`.
+
+### Context of EFX expressions
+In the various properties that contain an EFX expression, the expression now include the context declaration.
+
+### Simplification of field types
+The field type `internal-code` is replaced by `code`, as they have the same meaning.
+
+The field types `zoned-date` and `zoned-time` are replaced by `date` and `time`, as those types already require a time zone.
 
 ### Updated information
-The node identifiers have been changed, to make them more recognizable: NB-Lot, ND-ContractingParty, etc.
+New fields and nodes have been added.
 
 Various constraints have been added and updated.
 
+The information on the XML structure is now at the start of the file.
+
 ### Corrections
-The repeatability of several fields and nodes was corrected.
+Incorrect default values for the `mandatory` property have been corrected.
 
-The XPath of 2 fields was corrected (issue #27, thanks to @fpoort for reporting this).
+
+## Notice types
+The `notice-types` folder now contains an index file named `notice-types.json`, which describes each available notice type.
 
 
 ## View templates
-All view templates have been updated to use the new node identifiers, along with various small updates and corrections. 
+The `view-templates` folder now contains an index file named `view-templates.json`, which describes each available template.
+
+All view templates have been updated with various small updates and corrections. 
 
 :grey_exclamation: _The content of the view templates is still a work in progress. Subsequent releases of the SDK will gradually provide you with better and better view templates as we spot and correct mistakes._
 
 
 ## Translations
-A new `group` asset type was added, with translations on in English for now.
+The asset types and label types used to identify translations have been changed, to improve consistency and clarity.
+Asset types:
+* `decoration` to `auxiliary`
+* `condition` to `expression`
+* `business_term` to `business-term`
+
+Label types:
+* `value` to `name`
+* `message_template` to `template`
+* `message` to `text`
+
+Labels for codelist names, notice types, and view template names have been added.
 
-Translations for several items in codelists, fields and rules have been updated and corrected.
+Several translations have been updated and corrected.

README.md

@@ -1,26 +1,35 @@
-➤ **[Release notes for the latest version](CHANGELOG.md)** 
+**[:memo: Latest Release Notes](CHANGELOG.md)** | **[:package: Latest Release Artifacts](https://search.maven.org/search?q=g:%22eu.europa.ted.eforms%22%20AND%20a:%22eforms-sdk%22)**
 
-# Disclaimer
-This project is currently under development. We may update the current version and replace it with a new version at any time. In this case, we recommend you retrieve the latest version in order to ensure technical compliance. The Publications Office cannot guarantee the accuracy, adequacy, validity, reliability, availability or completeness of this information and accepts no responsibility for any use you may make of this project’s component parts.
+---
+
+_**Disclaimer:** This project is currently under development. We may update the current version and replace it with a new version at any time. In this case, we recommend you retrieve the latest version in order to ensure technical compliance. The Publications Office cannot guarantee the accuracy, adequacy, validity, reliability, availability or completeness of this information and accepts no responsibility for any use you may make of this project’s component parts._
+
+---
 
 # Summary
-This project is managed by the Publications Office of the European Union to enable implementation of the European legislation for publication of notices for public procurement on the [TED website](https://ted.europa.eu/TED/). For more information, see the [DG GROW eForms](https://ec.europa.eu/growth/single-market/public-procurement/digital/eforms_en) page.
+This project is managed by the Publications Office of the European Union to enable implementation of the European legislation for publication of notices for public procurement on the [TED website](https://ted.europa.eu/TED/). For more information, see the [DG GROW eForms](https://ec.europa.eu/growth/single-market/public-procurement/digital/eforms_en) page. 
 
 This project aims to provide the necessary tools for being able to build valid eForms notices.
-It is composed of the following:
-- `codelists`: This folder contains the codelists used in eForms notices, as [Genericode files](https://docs.oasis-open.org/codelist/genericode/doc/oasis-code-list-representation-genericode.html).
-- `schemas`: This folder contains a provisional release of the schemas. There are based on Pre-Award document types of the [UBL 2.3 standard](https://docs.oasis-open.org/ubl/UBL-2.3.html).
+It is composed of the following folders:
+- `codelists`: The codelists used in eForms notices, as [Genericode files](https://docs.oasis-open.org/codelist/genericode/doc/oasis-code-list-representation-genericode.html).
+- `schemas`: The XML schemas used for eForms notices. There are based on Pre-Award document types of the [UBL 2.3 standard](https://docs.oasis-open.org/ubl/UBL-2.3.html).
 The adopted approach is to keep the whole set of UBL elements but only support the use of a subset. It provides the advantage of having the rules outside of the schema. Adding extra fields for which a UBL element already exists will not require the definition of a new schema.
-- `efx-grammar`: This folder contains the ANTLR grammar for the eForms expression language (EFX).
-- `schematrons`: This folder contains Schematron rules which are used to check the validity of notices as per the eForms regulation. Any kind of rule or constraint is implemented in Schematron.
-- `examples`: This folder contains some notice examples together with their validation report in SVRL.
-- `notice-types`: This folder contains a definition of each type of notice, in JSON.
-- `fields`: This folder contains information on the fields that compose an eForms notice.
-- `view-templates`: This folder contains structured information on the visualisation of notices.
-- `translations`: This folder contains translations of various labels and short texts used in eForms notices.
+- `efx-grammar`: The ANTLR grammar for the eForms expression language (EFX).
+- `schematrons`: The Schematron rules used to check the validity of notices as per the eForms regulation. Any kind of rule or constraint is implemented in Schematron.
+- `examples`: Examples of eForms notices together with their validation report in SVRL.
+- `notice-types`: Definitions of each type of notice, in JSON.
+- `fields`: Information on the fields that compose an eForms notice.
+- `view-templates`: Structured information on the visualisation of notices.
+- `translations`: Translations of various labels and short texts used in eForms notices.
+
+You can download the latest release from Maven Central.
 
-The documentation is available at https://docs.ted.europa.eu.
+[![Maven Central](https://img.shields.io/maven-central/v/eu.europa.ted.eforms/eforms-sdk.svg?label=Download)](https://search.maven.org/search?q=g:%22eu.europa.ted.eforms%22%20AND%20a:%22eforms-sdk%22)
+
+The documentation is available at https://docs.ted.europa.eu/eforms/latest.
 
 # Versioning
 
+The eForms SDK uses semantic versioning. For more information: https://docs.ted.europa.eu/eforms/latest/versioning
+
 Provisional releases of the eForms schema and documentation that were provided during 2020 via SIMAP had a different versioning scheme and are replaced with this SDK that combines them into one bundle with one version number.

codelists/accessibility.gc

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!--File generated from metadata database version 0.5.0 created on the 2022-07-08T11:00-->
+<!--File generated from metadata database version 1.0.0 created on the 2022-08-05T10:24:40-->
 <gc:CodeList xmlns:gc="http://docs.oasis-open.org/codelist/ns/genericode/1.0/">
    <Identification>
       <ShortName>Accessibility</ShortName>

codelists/applicability.gc

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!--File generated from metadata database version 0.5.0 created on the 2022-07-08T11:00-->
+<!--File generated from metadata database version 1.0.0 created on the 2022-08-05T10:24:40-->
 <gc:CodeList xmlns:gc="http://docs.oasis-open.org/codelist/ns/genericode/1.0/">
    <Identification>
       <ShortName>Applicability</ShortName>

codelists/applicability_reserved-execution.gc

@@ -1,12 +1,12 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!--File generated from metadata database version 0.5.0 created on the 2022-07-08T11:00-->
+<!--File generated from metadata database version 1.0.0 created on the 2022-08-05T10:24:40-->
 <gc:CodeList xmlns:gc="http://docs.oasis-open.org/codelist/ns/genericode/1.0/">
    <Identification>
       <ShortName>ReservedExecution</ShortName>
       <LongName>reserved-execution</LongName>
       <LongName Identifier="listId">http://publications.europa.eu/resource/authority/applicability</LongName>
       <LongName Identifier="eFormsParentId">applicability</LongName>
-      <Version>0.5.0</Version>
+      <Version>1.0.0</Version>
       <CanonicalUri/>
       <CanonicalVersionUri/>
       <Agency>

codelists/award-criterion-type.gc

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!--File generated from metadata database version 0.5.0 created on the 2022-07-08T11:00-->
+<!--File generated from metadata database version 1.0.0 created on the 2022-08-05T10:24:40-->
 <gc:CodeList xmlns:gc="http://docs.oasis-open.org/codelist/ns/genericode/1.0/">
    <Identification>
       <ShortName>AwardCriterionType</ShortName>

codelists/buyer-categories.gc

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!--File generated from metadata database version 0.5.0 created on the 2022-07-08T11:00-->
+<!--File generated from metadata database version 1.0.0 created on the 2022-08-05T10:24:40-->
 <gc:CodeList xmlns:gc="http://docs.oasis-open.org/codelist/ns/genericode/1.0/">
    <Identification>
       <ShortName>BuyerCategories</ShortName>

codelists/buyer-contracting-type.gc

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!--File generated from metadata database version 0.5.0 created on the 2022-07-08T11:00-->
+<!--File generated from metadata database version 1.0.0 created on the 2022-08-05T10:24:40-->
 <gc:CodeList xmlns:gc="http://docs.oasis-open.org/codelist/ns/genericode/1.0/">
    <Identification>
       <ShortName>BuyerContractingType</ShortName>

codelists/buyer-legal-type.gc

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!--File generated from metadata database version 0.5.0 created on the 2022-07-08T11:00-->
+<!--File generated from metadata database version 1.0.0 created on the 2022-08-05T10:24:40-->
 <gc:CodeList xmlns:gc="http://docs.oasis-open.org/codelist/ns/genericode/1.0/">
    <Identification>
       <ShortName>BuyerLegalType</ShortName>

codelists/change-corrig-justification.gc

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!--File generated from metadata database version 0.5.0 created on the 2022-07-08T11:00-->
+<!--File generated from metadata database version 1.0.0 created on the 2022-08-05T10:24:40-->
 <gc:CodeList xmlns:gc="http://docs.oasis-open.org/codelist/ns/genericode/1.0/">
    <Identification>
       <ShortName>ChangeCorrigJustification</ShortName>

codelists/classification-type.gc

@@ -1,5 +1,5 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<!--File generated from metadata database version 0.5.0 created on the 2022-07-08T11:00-->
+<!--File generated from metadata database version 1.0.0 created on the 2022-08-05T10:24:40-->
 <gc:CodeList xmlns:gc="http://docs.oasis-open.org/codelist/ns/genericode/1.0/">
    <Identification>
       <ShortName>ClassificationType</ShortName>