Acknowledgements
+Acknowledgements
Former and current members of the EpiForecasts group (https://epiforecasts.io):
-Akira Endo, Alexis Robert, Ciara McCarthy, Hannah Choi,
-Hugo Gruson, James Azam, James Munday, Joel Hellewell,
-Joseph Palmer, Kath Sherratt, Liza Hadley, Manuel Stapper,
-Nikos Bosse, Robin Thompson, Sam Abbott, Sophie Meakin,
-Toshiaki Asakura
+Akira Endo, Alexis Robert, Ciara McCarthy, Friederike Becker,
+Hannah Choi, Hugo Gruson, James Azam, James Munday,
+Joel Hellewell, Joseph Palmer, Kath Sherratt, Liza Hadley,
+Manuel Stapper, Nikos Bosse, Robin Thompson, Sam Abbott,
+Sophie Meakin, Toshiaki Asakura
Collaborators at LSHTM and elsewhere.
Models are a tool to combine data (what we know) with assumptions and theory (what we think) to learn about what we don’t know.
-
When data is abundant, models and analytics can generate insight without many additional assumptions.
-
When data is sparse (e.g. early in an outbreak), modellers need to make more assumptions to generate insights.
-
January 2020: Can COVID-19 be controlled by contact tracing?
-
Probability of control depends on intensity of transmission and contact tracing effort.
-
“We illustrate the potential impact that flawed model inferences can have on public health policy with the model described […] by Joel Hellewell and colleagues, which is part of the scientific evidence informing the UK Government’s response to COVID-19.”
@@ -527,8 +527,8 @@“All models are wrong, but some are useful”
@@ -540,8 +540,8 @@“All models are wrong, but some are useful”
@@ -558,30 +558,30 @@The future as a (particular) data gap
+The future as a (particular) data gap
-
The future as a (particular) data gap
+The future as a (particular) data gap
-
Mechanistic models support causal understanding, but predictions can have value in their own right
+Mechanistic models support causal understanding, but predictions can have value in their own right
-
Mechanistic models support causal understanding, but predict
-
-Short-term forecasts can inform decision making
+
+Short-term forecasts can inform decision making
-
+
@@ -629,10 +629,10 @@ Short-term forecasts can inform decision making
-
-Short-term forecasts can inform decision making
+
+Short-term forecasts can inform decision making
-
+
@@ -643,10 +643,10 @@ Short-term forecasts can inform decision making
-
-Short-term forecasts can inform decision making
+
+Short-term forecasts can inform decision making
-
+
@@ -657,25 +657,25 @@ Short-term forecasts can inform decision making
-
-Forecasting Ebola
-
+
+Forecasting Ebola
+
-
-Ebola in West Africa, 2013-16
+
+Ebola in West Africa, 2013-16
-
+
-
-
+
+
“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling, or levelling off?”
@@ -688,10 +688,10 @@
-
-
+
+
-
+
@@ -704,10 +704,10 @@
-
-Forecasts can be assessed/validated
+
+Forecasts can be assessed/validated
-
+
@@ -719,8 +719,8 @@ Forecasts can be assessed/validated
-
-Forecasting paradigm
+
+Forecasting paradigm
“maximise sharpness subject to calibration”
@@ -733,10 +733,10 @@ Forecasting paradigm
-
-Ebola: how wrong were our models?
+
+Ebola: how wrong were our models?
-
+
@@ -749,31 +749,31 @@ Ebola: how wrong were our models?
-
-
+
+
Ebola forecasts could be trusted for up to 2 weeks
-
-
+
+
Our Ebola forecasts could be trusted for up to 2 weeks
-
-Forecasting COVID-19
-
+
+Forecasting COVID-19
+
-
-Forecasting via the renewal equation
+
+Forecasting via the renewal equation
\begin{align}
\textrm{New infections}~I(t) & = R_t \sum_{\tau} g_{\tau} I_{t-\tau}\\
@@ -792,10 +792,10 @@ Forecasting via the renewal equation
-
-Global COVID case forecasts via the renewal equation
+
+Global COVID case forecasts via the renewal equation
-
+
@@ -823,10 +823,10 @@ Global COVID case forecasts via the renewal equation
-
-Forecasting to inform policy in the UK
+
+Forecasting to inform policy in the UK
-
+
@@ -841,10 +841,10 @@ Forecasting to inform policy in the UK
-
-European COVID-19 Forecast Hub
+
+European COVID-19 Forecast Hub
-
+
@@ -872,15 +872,15 @@ European COVID-19 Forecast Hub
-
-How good were COVID forecasts?
-
+
+How good were COVID forecasts?
+
-
-
+
+
We can compare forecasts using
proper scoring rules
\[\mathrm{CRPS}(F, x) = \mathbb{E}|X-x| - \frac{1}{2}\mathbb{E}|X-X'|\]
@@ -900,10 +900,10 @@
-
-Median ensemble outperforms individual models
+
+Median ensemble outperforms individual models
-
+
@@ -915,8 +915,8 @@ Median ensemble outperforms individual models
-
-
+
+
We can compare forecasts using proper scoring rules
\[\mathrm{CRPS}(F, x) = \mathbb{E}|X-x| - \frac{1}{2}\mathbb{E}|X-X'|\]
@@ -925,10 +925,10 @@
-
-Absolute quality of forecasts #1: baseline models
+
+Absolute quality of forecasts #1: baseline models
-
+
@@ -942,10 +942,10 @@ Absolute quality of forecasts #1: baseline models
-
-Absolute quality of forecasts #2: calibration
+
+Absolute quality of forecasts #2: calibration
-
+
@@ -957,8 +957,8 @@ Absolute quality of forecasts #2: calibration
-
-What limits predictive ability?
+
+What limits predictive ability?
- Unpredictable human behaviour?
- Unpredictable pathogen biology?
@@ -968,10 +968,10 @@ What limits predictive ability?
-
-Unpredictable human behavoiur?
+
+Unpredictable human behavoiur?
-
+
@@ -983,13 +983,13 @@ Unpredictable human behavoiur?
-
-
+
+
Observed behaviour as predictor: improvement of forecasts, but only once age-specific reporting is taken into account.
-
+
@@ -1001,10 +1001,10 @@
-
-Unpredictable biology?
+
+Unpredictable biology?
-
+
@@ -1016,10 +1016,10 @@ Unpredictable biology?
-
-Variants as predictor: improvements of forecasts during transitions
+
+Variants as predictor: improvements of forecasts during transitions
-
+
@@ -1032,10 +1032,10 @@ Variants as predictor: improvements of forecasts during tran
-
-Bad models? Human vs. machine.
+
+Bad models? Human vs. machine.
-
+
@@ -1047,8 +1047,8 @@ Bad models? Human vs. machine.
-
-
+
+
Humans better than models at predicting cases, but not deaths
@@ -1061,8 +1061,8 @@
-
-
+
+
Amongst models, ones that focus on a single country tended to do better
@@ -1075,10 +1075,10 @@
-
-Inherent limits?
+
+Inherent limits?
-
+
@@ -1091,15 +1091,15 @@ Inherent limits?
-
-What can we conclude for the next pandemic?
-
+
+What can we conclude for the next pandemic?
+
-
-Summary
+
+Summary
- Covid-19 forecasts have been relatively poor further than one or two generations ahead
- Ensembles perform best, but can be difficult to interpret
@@ -1108,8 +1108,8 @@ Summary
-
-Open questions:
+
+Open questions:
- Can predictive performance be improved?
- Are we measuring predictive performance in the right way?
@@ -1118,13 +1118,13 @@ Open questions:
-
-
+
+
Alternative ways of measuring predictive performance change the ranking of models.
-
+
@@ -1137,10 +1137,10 @@
-
-Forecasting and nowcasting remain relevant
+
+Forecasting and nowcasting remain relevant
-
+
@@ -1153,20 +1153,20 @@ Forecasting and nowcasting remain relevant
-
-New initiatives
+
+New initiatives
-
+
-
-European respiratory hub
+
+European respiratory hub
-
+
@@ -1178,10 +1178,10 @@ European respiratory hub
-
-Evaluating conditional forecasts (“scenarios”)
+
+Evaluating conditional forecasts (“scenarios”)
-
+
@@ -1194,8 +1194,8 @@ Evaluating conditional forecasts (“scenarios”)
-
-We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools
+
+We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools
-
-“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling or levelling off?”
+
+“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling or levelling off?”
– Hans Rosling, Liberia, 2014
@@ -1221,10 +1221,10 @@ “We were losing ourselves in details […] all we
-
-Slides at
+
+Slides at
-
+
Short-term forecasts can inform decision making
+Short-term forecasts can inform decision making
-
Short-term forecasts can inform decision making
Short-term forecasts can inform decision making
+Short-term forecasts can inform decision making
-
Short-term forecasts can inform decision making
Short-term forecasts can inform decision making
+Short-term forecasts can inform decision making
-
Short-term forecasts can inform decision making
Forecasting Ebola
-Forecasting Ebola
+Ebola in West Africa, 2013-16
+Ebola in West Africa, 2013-16
-
“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling, or levelling off?”
@@ -688,10 +688,10 @@
Forecasts can be assessed/validated
+Forecasts can be assessed/validated
-
Forecasts can be assessed/validated
Forecasting paradigm
+Forecasting paradigm
“maximise sharpness subject to calibration”
@@ -733,10 +733,10 @@Forecasting paradigm
Ebola: how wrong were our models?
+Ebola: how wrong were our models?
-
Ebola: how wrong were our models?
Ebola forecasts could be trusted for up to 2 weeks
Our Ebola forecasts could be trusted for up to 2 weeks
Forecasting COVID-19
-Forecasting COVID-19
+Forecasting via the renewal equation
+Forecasting via the renewal equation
Forecasting via the renewal equation
Global COVID case forecasts via the renewal equation
+Global COVID case forecasts via the renewal equation
-
Global COVID case forecasts via the renewal equation
Forecasting to inform policy in the UK
+Forecasting to inform policy in the UK
-
Forecasting to inform policy in the UK
European COVID-19 Forecast Hub
+European COVID-19 Forecast Hub
-
European COVID-19 Forecast Hub
How good were COVID forecasts?
-How good were COVID forecasts?
+
We can compare forecasts using
proper scoring rules
\[\mathrm{CRPS}(F, x) = \mathbb{E}|X-x| - \frac{1}{2}\mathbb{E}|X-X'|\]
@@ -900,10 +900,10 @@
Median ensemble outperforms individual models
+Median ensemble outperforms individual models
-
Median ensemble outperforms individual models
We can compare forecasts using proper scoring rules \[\mathrm{CRPS}(F, x) = \mathbb{E}|X-x| - \frac{1}{2}\mathbb{E}|X-X'|\] @@ -925,10 +925,10 @@
Absolute quality of forecasts #1: baseline models
+Absolute quality of forecasts #1: baseline models
-
Absolute quality of forecasts #1: baseline models
Absolute quality of forecasts #2: calibration
+Absolute quality of forecasts #2: calibration
-
Absolute quality of forecasts #2: calibration
What limits predictive ability?
+What limits predictive ability?
- Unpredictable human behaviour?
- Unpredictable pathogen biology? @@ -968,10 +968,10 @@
What limits predictive ability?
Unpredictable human behavoiur?
+Unpredictable human behavoiur?
-
Unpredictable human behavoiur?
Observed behaviour as predictor: improvement of forecasts, but only once age-specific reporting is taken into account.
-
Unpredictable biology?
+Unpredictable biology?
-
Unpredictable biology?
Variants as predictor: improvements of forecasts during transitions
+Variants as predictor: improvements of forecasts during transitions
-
Variants as predictor: improvements of forecasts during tran
-
-Bad models? Human vs. machine.
+
+Bad models? Human vs. machine.
-
+
@@ -1047,8 +1047,8 @@ Bad models? Human vs. machine.
-
-
+
+
Humans better than models at predicting cases, but not deaths
@@ -1061,8 +1061,8 @@
-
-
+
+
Amongst models, ones that focus on a single country tended to do better
@@ -1075,10 +1075,10 @@
-
-Inherent limits?
+
+Inherent limits?
-
+
@@ -1091,15 +1091,15 @@ Inherent limits?
-
-What can we conclude for the next pandemic?
-
+
+What can we conclude for the next pandemic?
+
-
-Summary
+
+Summary
- Covid-19 forecasts have been relatively poor further than one or two generations ahead
- Ensembles perform best, but can be difficult to interpret
@@ -1108,8 +1108,8 @@ Summary
-
-Open questions:
+
+Open questions:
- Can predictive performance be improved?
- Are we measuring predictive performance in the right way?
@@ -1118,13 +1118,13 @@ Open questions:
-
-
+
+
Alternative ways of measuring predictive performance change the ranking of models.
-
+
@@ -1137,10 +1137,10 @@
-
-Forecasting and nowcasting remain relevant
+
+Forecasting and nowcasting remain relevant
-
+
@@ -1153,20 +1153,20 @@ Forecasting and nowcasting remain relevant
-
-New initiatives
+
+New initiatives
-
+
-
-European respiratory hub
+
+European respiratory hub
-
+
@@ -1178,10 +1178,10 @@ European respiratory hub
-
-Evaluating conditional forecasts (“scenarios”)
+
+Evaluating conditional forecasts (“scenarios”)
-
+
@@ -1194,8 +1194,8 @@ Evaluating conditional forecasts (“scenarios”)
-
-We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools
+
+We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools
-
-“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling or levelling off?”
+
+“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling or levelling off?”
– Hans Rosling, Liberia, 2014
@@ -1221,10 +1221,10 @@ “We were losing ourselves in details […] all we
-
-Slides at
+
+Slides at
-
+
Bad models? Human vs. machine.
+Bad models? Human vs. machine.
-
Bad models? Human vs. machine.
Humans better than models at predicting cases, but not deaths
@@ -1061,8 +1061,8 @@
Amongst models, ones that focus on a single country tended to do better
@@ -1075,10 +1075,10 @@
Inherent limits?
+Inherent limits?
-
Inherent limits?
What can we conclude for the next pandemic?
-What can we conclude for the next pandemic?
+Summary
+Summary
- Covid-19 forecasts have been relatively poor further than one or two generations ahead
- Ensembles perform best, but can be difficult to interpret @@ -1108,8 +1108,8 @@
Summary
Open questions:
+Open questions:
- Can predictive performance be improved?
- Are we measuring predictive performance in the right way? @@ -1118,13 +1118,13 @@
Open questions:
Alternative ways of measuring predictive performance change the ranking of models.
-
Forecasting and nowcasting remain relevant
+Forecasting and nowcasting remain relevant
-
Forecasting and nowcasting remain relevant
New initiatives
+New initiatives
-
European respiratory hub
+European respiratory hub
-
European respiratory hub
Evaluating conditional forecasts (“scenarios”)
+Evaluating conditional forecasts (“scenarios”)
-
Evaluating conditional forecasts (“scenarios”)
-
-We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools
+
+We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools
-
-“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling or levelling off?”
+
+“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling or levelling off?”
– Hans Rosling, Liberia, 2014
@@ -1221,10 +1221,10 @@ “We were losing ourselves in details […] all we
-
-Slides at
+
+Slides at
-
+
We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools
+We need collaborative efforts, using standardised datasets to compare methods and generating sustainable tools
“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling or levelling off?”
+“We were losing ourselves in details […] all we needed to know is, are the number of cases rising, falling or levelling off?”
– Hans Rosling, Liberia, 2014
@@ -1221,10 +1221,10 @@“We were losing ourselves in details […] all we
Slides at
+Slides at
-