Sacred Integration for Reproducibility

[vlomonaco]

At the moment Avalanche have a little support for experiments reproducibility. This can be done mostly by config files, logs and a little support form the benchmarks.

I was thinking it would nice to integrate those mechanisms with Sacred. It is a great tool for reproducibility. What do you think?

[AntonioCarta]

I know it and I think it's useful but I never used it. I would not add anything new to the Evaluation/Metrics side for the alpha. First, we have to decide how to do it.

Also, is this something that needs to be done by a CL library? Personally, I would not use a library that forced me to a specific tool to track my own experiments because I already have my tools. Basically, I think avalanche should be a library and not a framework.

[AndreaCossu]

I agree with @AntonioCarta on this. What we could provide are utilities to launch experiment from configuration file (e.g. yaml) and complete logs and, when needed, plots. I don't think it is required to have all this stuff ready for the alpha but most definitely they will be needed in future releases.

[vlomonaco]

Yes, I agree with both of you! But when developing we should keep in mind that people may need to use it. So we need to carefully plan how to make the integration of the two tools as easy as possible.