Simulation calibration

[seabbs]

Check the model works by doing the following

• Generate datasets from the prior
• Fit the model to these datasets
• Compare prior and posterior ranks

See stantargets for a fairly simplistic but easy to implement version of this (with data generation handled by the stan model rather than a native R implementation): https://docs.ropensci.org/stantargets/articles/simulation.html

[seabbs]

Linked to #7

[seabbs]

Resources:

https://mc-stan.org/docs/2_23/stan-users-guide/simulation-based-calibration.html
https://arxiv.org/abs/1804.06788
https://jasonmtroos.github.io/sbcrs/
https://github.com/hyunjimoon/SBC

[seabbs]

Many of the current tool sets for this rely on the model being implemented in R as well as in stan which is quite effort-intensive. Instead, I have written a function to simulate from the stan model (very inefficiently).

Not clear which of the tool sets are going to be the easiest to use with this (and several (i.e SBC) are under very active development. Will test out a few and see what can be done most easily and ideally with minimal compute overhead.