Checks for scenario hub

[kathsherratt]

We may need to edit some of the validations to accept scenario hub submissions.

For now I can think of:

• csv files
  • must include all of {1:16} weeks ahead, optionally {17:52} weeks ahead
  • accept origin_date instead of / as well as forecast_date
  • scenario code present in each row and among the accepted list of codes. For round 0 these are here: https://github.com/midas-network/covid19-scenario-modeling-hub#submission-information
• Metadata includes extra fields
• Team folder can include an Abstract.md file (although maybe the check shouldn't fail if there isn't one). (Not sure if our current checks flag if there are non-csv or metadata files at the moment so this may not need any action.)

[Bisaloo]

Most of this is already implemented in the schema files:

• https://github.com/covid19-forecast-hub-europe/covid19-scenario-hub-europe/blob/main/data-processed/schema-metadata.yml
• https://github.com/covid19-forecast-hub-europe/covid19-scenario-hub-europe/blob/main/data-processed/schema-data.yml

I want to try to do a test submission this week but I have been too busy until now.

[kathsherratt]

Great that this is mostly implemented!

I was wondering if it would help to have for each round to have a round-specific metadata or config type file that could be used for these validations.

This would include checks that need to happen for each forecast file but are round specific. For instance, the scenario IDs, accepted start/end dates for projections, and the number of weeks allowed should all be checked against a pre-specified list of valid values, but these values can/will each change for each round. I guess we wouldn't want to have to edit a basic schema files each time for these.

I am not sure how difficult it would be to add this functionality to the package, and there could be a better way of doing this - what do you think?

[Bisaloo]

Since the schema files are located in the repo (covid19-forecast-hub-europe/covid19-scenario-hub-europe in this case), I believe it's fine to edit the relevant lines in the schema files each time.

If we go this route, then no change in required in the ForecastHubValidations package.