How to measure quality of dataset/models in the online public catalog?

[acocac]

There was a very interesting question in a recent presentation of scivision about how will we monitor or ensure the quality of the datasets/models pushed to the public catalog?

Apart from double checking URLs using pydantic, it would be great to hear potential ideas to be considered in future releases.

[acocac]

A potential indicator is to set related scientific publications. This can be added as an additional key in model/dataset catalogs.

[ots22]

It is probably worth periodically checking that the datasources are reachable and that the models can be installed and run - perhaps if a URL doesn't resolve after a few attempts, we e.g. automatically open a PR that sets an appropriate flag in the catalog entry, or removes them altogether.