Transitioning from the European COVID19 Forecast Hub

Changes to team submissions

Below are the modifications from the existing European forecast hub, to the US Scenario hub structure, for each file that teams are asked to submit. Includes changes to file names, formatting, and validation checks, for model output, metadata, and the (new addition) abstract.

Model output

• Weekly submission file
  • Format keeps csv, using zip files if size exceeds 100MB
  • File name keeps YYYY-MM-DD-team-model.csv
  • Included fields:
    • Replaces:
      • forecast_date with origin_date
      • target with target_variable and horizon
    • Keeps all other existing fields, with the same formatting (target_end_date, location, type, value, quantile)
    • Adds
      • scenario_id - a string in the format letter-YYYY-MM-DD, specified in README
• Validation
  • Adds checks scenario_id matches correct format
  • Modifies horizon allows 1 through 26 weeks ahead; suggested weeks to include earliest 13 weeks ahead and furthest 26 weeks ahead

Metadata

• Metadata text file
  • Format keeps .txt with YAML format
  • Naming keeps team-model-metadata.txt
  • Included fields:
    • Keeps all existing fields in forecast hub
    • Adds fields (see description)
      • model_version, modelling_NPI, compliance_NPI, contact_tracing, testing, vaccine_efficacy_transmission, vaccine_efficacy_delay, vaccine_hesitancy, vaccine_immunity_duration, natural_immunity_duration, case_fatality_rate *, infection_fatality_rate *, asymptomatics *, age_groups, importations, confidence_interval_method, calibration, spatial_structure, data_inputs
• Validation
  • Adds checks that all added fields above are strings, except the numeric fields, marked by (*). String is allowed to be "Not applicable"

Abstract

• Adds new file added as of round 12, suggested for all teams (see example)
  • Naming: YYYY-MM-DD-team-model-abstract.md
  • Format: Free text markdown with suggested headers
    • Summarise results
    • Explain results
    • Describe model assumptions (specifically: susceptibility, transmissibility, generation time, waning immunity)
    • Describe changes from previous rounds