-
Notifications
You must be signed in to change notification settings - Fork 0
/
about.Rmd
146 lines (91 loc) · 8.64 KB
/
about.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
---
title: "Background"
output:
html_document:
toc: false
toc_float: false
---
The European Scenario Hub brings together scenario modellers across Europe. We work collaboratively to:
- Inform short- and mid-term policy strategies for managing COVID-19 across Europe
- Better understand the drivers of possible COVID-19 futures
We combine and interpret the work of many modelling teams, focussing on cases, deaths, and hospitalisations over the next 12 months in 32 European countries.
# {.tabset}
## What?
---
##### Key definitions
**Scenarios** _isolate fundamental drivers of COVID-19 dynamics_
**Models** _quantitatively encode the influence of these drivers on COVID-19 outcomes_
**Projections** _demonstrate the relative impact of different variables, given current knowledge_
---
##### Defining scenario modelling
Scenario models for COVID-19 aim to bound the uncertainty around future outcomes over some time frame. Each potential scenario is composed of one or more variables (which could be, for example, biological parameters such as the rate of immunity waning, or assumed future changes in contact behaviours) deemed plausible. The scenario models encode the influence of these variables on COVID-19 outcomes, and results from scenario models show possible outcomes within the given timeframe (assuming that present causal relationships hold in the future).
##### Using scenarios
The definition of scenario variables and values depends on the intended use of scenario results. One purpose of scenario modelling is to support comparisons of possible actions taken in the present with long term effects. Separately, scenarios can also be used to understand the sensitivity of future outcomes to varying parameter conditions.
##### Interpreting scenarios
Each single model provides a framework for estimating the upper and lower limits of uncertainty in COVID-19 outcomes corresponding to each scenario. However, there is no objectively best framework for modelling the transmission of SARS-CoV-2 and resulting dynamics of COVID-19, meaning the true range of uncertainty may be most appropriately expressed by results from multiple alternative models.
## Why?
The short-term goals of the European Scenario Hub are:
1. Creation and visualisation of open access scenario projections
2. Collaboration among hub participants (that improves the quality of projections)
3. Identifiable potential for use of results in policy decision-making
4. Ethical communication of scenario projections to the public;
5. Secondary analysis of scenario hub projections including
* Degree of agreement between different models for the same scenario
* Understanding the impact of choices of model structure and complexity on projected futures
* Methods for combining scenario results
Long term goals include:
6. Creating an open database of standardised, comparable scenario projections that are easily accessible for secondary analyses by the wider research community
7. Identifying benefits or drawbacks of collaborative improve real-time scenario modelling with implications for future work
8. Creating a self-sustaining community of European scenario modellers
On this page we address the first four of the short-term aims of the Scenario Hub.
## How? {.tabset .tabset-pills}
### Identifying scenarios and setting parameters
We will create scenarios at 4-8 week intervals with each set of scenarios (a round) projecting over a future 3-12 months. As a starting point, our framework for setting scenarios in each round is a 2x2 matrix composed of two variables, each with optimistic and pessimistic parameter values for its influence on COVID-19 outcomes.
#### Variables
We aim to include one variable that is biologically intrinsic to COVID-19, and one variable that can be influenced extrinsically by national-level policy action. For each variable we will identify one or more observable parameters that represent the causal pathway from the variable to the outcome.
For each scenario round, we will collaborate with the ECDC to define the extrinsic variable, based on their interactions with national policy-makers and assessment of possible policy options. We will draw on the epidemiology of COVID-19 in Europe to suggest themes for the biologically intrinsic variable. After an initial consultation with the ECDC, we will share and seek suggestions for the scenario variables from participating teams.
#### Parameters
Each of the two variables can be expressed using one or multiple parameters representing an observable value on the causal pathway between the variable and outcome. We can explore the uncertain impact of each parameter by identifying the two most plausible parameter values that would create either the best possible (optimistic) or worst possible (pessimistic) COVID-19 outcome.
Modelling teams do not have to include the underlying variables in their model, but the model structure should be able to include each explicit parameter value. Additional implicit parameters may be relevant to each variable but not quantitatively specified in the scenario definition. Whether these parameters are included in a model is at each teams’ discretion.
For each scenario round, after defining the variables we will collaborate with teams to define which parameters are explicitly specified and at what value. Teams must be able to include the explicit parameters, and we do not specify any other restrictions on the type or structure of models that teams use.
### Contributing and combining scenario projections
We focus on weekly incident targets, for any of the following COVID-19 outcomes: infections, cases, and deaths. To align with other Hubs we ask for quantile intervals at 0.01, 0.025, 0.05, every 5% to 0.95, 0.975, and 0.99. Inclusion in ensemble models requires a full set of quantiles from 0.01 to 0.99. We will collect metadata from teams in the form of a structured abstract. This will vary slightly with each round of scenarios. See the project Wiki for specific details.
We will create ensembles from the individual models for each scenario round separately. As an initial choice we will create a median ensemble by taking the median of the given predictive quantiles of the different models for each scenario. Other ensemble choices will be considered and differences between outcomes presented by different models discussed with the modellers in order to avoid hiding differences from structural or parametric assumptions by taking the ensemble.
### Reporting and interpreting projections
For each scenario round, we will create a standardised visualisation allowing all models’ scenario projections to be compared. This will be presented on the website and we will follow this with a narrative summary of model results, including information taken from model abstracts to demonstrate differences between teams’ approach to and results from scenarios.
We aim to choose plausible scenarios that can inform policy but do not expect any of them to be an exact characterisation of the future. We therefore do not expect data to align with any one model result. Moreover, observed data would not align with any scenario output if the scenarios excluded a parameter that operated within the relevant timeframe and had substantial confounding effects across all of the scenarios.
## Related work
We also run the [European Forecasting Hub](https://covid19forecasthub.eu/), producing forecasts of COVID-19 across Europe over the next 1-4 weeks.
The European Scenario Hub and broader Hub approach follows work across Europe and the United States, including the:
- [US Scenario Modelling Hub](https://covid19scenariomodelinghub.org/)
- [US Forecast Hub](https://covid19forecasthub.org/)
- [German and Polish Forecast Hub](https://github.com/KITmetricslab/covid19-forecast-hub-de)
We work closely with these teams and we are grateful for their support in setting up European Hubs.
## The evaluation of the European Modelling Hubs
ECDC has conducted a survey among the participants of the European Modelling Hub, getting feedback and thus evaluating the European COVID-19 Forecast Hub, European COVID-19 Scenario Hub, and the European Modelling Hub meetings. The results and conclusions of the survey can be found below:
```{r load_libraries, echo = FALSE, warning = FALSE, message = FALSE}
library("yaml")
library("purrr")
library("fs")
library("dplyr")
```
```{r hub_evaluations, results='asis', echo=FALSE}
files <- rev(fs::dir_ls("hub_evaluations"))
dates <- sub(
"^([0-9]{4})-([0-9]{2})-([0-9]{2})-.*$",
"\\3/\\2/\\1",
basename(files)
)
nms <- files %>%
fs::path_file() %>%
fs::path_ext_remove() %>%
substring(12) %>%
{ gsub("-", " ", .) }
sprintf(
"- %s\t\t[%s](%s)",
dates,
nms,
files
) %>%
cat(sep = "\n")
```