Model checks against simulation

[seabbs]

Check the model works against some basic simulations generated in R.

Implement by:

• Adding a generative_model() function or similar for generating data

[jbracher]

For a paper one idea could be to simulate from models with the same parameters as obtained in the real-data study (assuming we cover different countries). Not entirely sure how to do that, though (I assume fixing parameters at their posterior means?).

[seabbs]

Yes good idea. I am looking at some simplistic simulation based calibration as a first step in.

[sbfnk]

In my understanding SBC is the more general approach - if doing that I don't think you need to bother with fixed parameters. If you're interested in particular parts of the parameter space you could subset to that.

[jbracher]

Not entirely sure what you mean here - what's SBC? (I found at least "Simulation-Based Calibration" and "Schwarz Bayesian Criterion" as vaguely plausible options ;) ).
Do you mean we just simulate from the full posterior of the model incl its observable time series? That sounds like a good solution to me.

[seabbs]

Yes I agree it is but there is an argument for testing performance explicitly under certain scenarios. That being said not planning on doing this here I think.

@jbracher : Yes that is the one. Quite a few resources out here but a few here:

https://mc-stan.org/docs/2_23/stan-users-guide/simulation-based-calibration.html
https://arxiv.org/abs/1804.06788
https://jasonmtroos.github.io/sbcrs/
https://github.com/hyunjimoon/SBC

Going to close this in favour of #3