How portable/deployable is the OpenSafely backend infrastructure?

[tombisho]

Hi,

I have previous talked with the team about the differences and synergies with DataSHIELD. I have worked through the getting started tutorial so I have some idea of how OpenSafely works.

My area of work is less about EHR and instead involves detailed epidemiological cohort studies. We have had some success with analysing cohort studies from across the world with DataSHIELD. The cohort data has remained with its custodians and we used DataSHIELD to take the analysis to the data. Relatively speaking, the infrastructure needed to deploy DataSHIELD is quite light weight and can be set up quite easily.

I would be interested in trying out OpenSafely in this different context by setting up a local instance to hold some demo study data From a brief look at the backend repositories, the deployment of the infrastructure looks more involved. Am I correct that it would be quite a lot of work to set up a small local instance to try it out? And by extension, it would be quite difficult to ask research partners to set up their own OpenSafely instance?

Depending on the answer to those questions, I was thinking about the possibility of collaborating with you to adapt our current infrastructure to run using the same model as OpenSafely on international cohort study data.

Specifically the intention would be to adhere to the OpenSafely principles where:

1. The user gets a container with a copy of the analysis environment with a synthetic copy of the real data
2. Results are checked for disclosure before being released
3. Code is public and reproducible

I look forward to hearing from you

Tom

[sebbacon]

Great - this is a use case that we plan to support. We have already run a proof of concept in a non-EHR backend, but as you've guessed, it is currently quite time-consuming. That's because until now OpenSAFELY has been built and optimised for our specific EHR/Primary Care/Covid19 requirements.

We are currently in the middle of piece of work to refactor the core cohort extraction code, to make it easier to support different kinds of backend data. We hope to complete an alpha version of this in about 4 weeks. Following that, we are planning a 6-week sprint to simplify deployment of new backends, with some form of pluggable infrastructure support (e.g. kubernetes support).

In short, it is possible right now, but you'd be better off waiting about 10 weeks, when it should be much easier. If you are able to describe your deployment environment (database; compute resources; operating systems; etc) we can feed your requirements into the deployment sprint.

[sebbacon]

Thanks. You've said there's usually MongoDB or MySQL. Is this usually accessed over ODBC? What operating systems and/or languages are typically available? Our current thinking is that deployment would usually be with Docker, which implies a Linux system with Docker installed must be available. Would this be a reasonable thing to expect / ask?

[tombisho]

I believe Opal (the data warehouse generally used with DataSHIELD) uses JDBC for MySQL and a Java driver for Mongo. I am afraid I am not an expert in the underlying mechanics of Opal....

We tend to encounter Ubuntu and Redhat most, and very occasionally Windows which tends not to go so well.

I think a Docker set up would be a good way to go. Indeed, this is the way things are headed with DataSHIELD because it handles all the dependencies etc. so much better. So typically we are now pointing people to these instructions instead of sending them a massive installation guide asking them to install Java DK, R and so on.

[tombisho]

Hi, I hope all the team have had a good summer. I just wanted to check back on the progress of the non-EHR backend for OpenSAFELY. I know this is probably quite soon after the proposed 10 weeks of work, but it would be useful to know where things have got to. As I said in my original message, it would be really useful to try managing our own instance that could host cohort data.

[sebbacon]

Hi, as you might expect we're not quite at a one-click install! But we have made some progress (the sprint is still ongoing). Will update you when we speak next.

[tombisho]

even 10 clicks would be a great achievement 😄 Yes I look forward to catching up