Provide data files externally to python package

[DavidLP]

To run unit tests and provide example data one needs to have rather large data files. These files should and cannot (max. quota ~ 250 MB) be stored in the python package hosted on pypi. I did not find a good solution to do this. The best would be a python package providing autodowload of files on demand for several supported file hosters. Otherwise github could be misused to a certain extend, by adding a data branch and downloading from there...

[DavidLP]

We have 150 Mb per branch (because of the data files) and 6 branches = 6 * 150 = 900 Mb. Github allows for 1 Gb before you get a "polite" E-Mail. Thus one more branch is not supported anymore and we have to find a solution asap. I will create a data branch within testbeam analysis and add a downloader tool for the exampels.

[DavidLP]

Did not come up with a good solution yet. Hosting at University is not a trivial task and you do not want to start username/psw access restrictions.

[DavidLP]

https://quiltdata.com/ look promising

[DavidLP]

Implemented in development

[DavidLP]

The Scibo campuscloud does not seem to be planned as a high availability cloud, frequent downtime of up to days (when it is weekend) are common. Additionally the downtime (an important classifier for an online hoster) seems not to be disclosed. This does not meet our requirementa, since the downtime stalls our tests and builds. We should search for a better service. I reopen this issue.

[themperek]

https://help.github.com/articles/distributing-large-binaries/ (seperate repo?)

[DavidLP]

This seems to be exactly what we want, even in same repo: https://git-lfs.github.com/

[laborleben]

This is maybe the only option.