README.md

collection

This repository is used to manage the resources displayed on bioimage.io.

Most users will not directly dispatch the workflows defined in this reporitory, but should instead login on bioimage.io and use the front-end to interact with the bioimage.io collection.

We currently do not have a workflow for direct upload, but publicly available resource packages may be staged with a stage workflow dispatch.

Maintaining the bioimage.io Collection

In order to update or add new resources to the bioimage.io collection, they have to undergo review. Current reviewers are listed in bioimageio_collection_config.json, under section reviewers.

Reviewer Onboarding

1. Open a pull request, adding a person to the list of reviewers, see #75 for an example.
   • one public email address is required
   • github id can be found using https://api.github.com/users/<github_username>
2. So the changes can be applied, the service needs to be restored manually. Reach out to the team or leave an issue.
3. Once the pull request has been merged, the new reviewer can
   • accept resource drafts
   • request changes on resource drafts
   • upload a new version for any resource

Adding a bioimage.io resource

A bioimage.io resource is created by a user uploading a resource package. Such a resource package could e.g. be a newly uploaded model package, or notebook package. Typically, uploaders would go via bioimage.io/upload. Alternatively, any direct link to a downloadable resource package (.zip-file) would work. The latter option is reserved for members of this repository (or the bioimageio org). Once available online the resource package is staged (see Staging section), tested (see Testing section), and reviewed (see Review section).

graph TD;
    stage[stage]
    test[test]
    cr[request changes]
    publish[accept/publish]
    backup[backup to Zenodo]

    stage-->test
    test-->cr
    cr-->stage
    test-->publish
    publish-->backup

Loading

Staging

If the resource package was uploaded via the bioimage.io website, the staging of the new resource draft is initiated automatically by the uploader service. Otherwise, given a download URL to a resource package, the stage workflow needs to be dispatched manually, or via github api ("run workflow")¹.

Staging unpacks the files from the zipped resource package to our public S3. Once unpacked, the staged resource draft is automatically tested (the test workflow is dispatched automatically at the end of the stage workflow).

Testing

Staged resource drafts are automatically tested:

• Is their metadata valid?
• Can test outputs be reproduced from test inputs?
• Are linked URLs available?
• ...

Tests can also be triggered (via github api or manually) by dispatching the test workflow².

Once the tests are completed, the uploader gets a notification via email that their draft is awaiting review; now a reviewer needs to take a look.

An overview of all pending resource drafts can be found at https://bioimageio-uploader.netlify.app/#/status. A draft is identified by its concept id (id from the rdf.yaml).

Review

Reviewers should check the models for technical correctness (aided by CI, see Testing section) and contents/metadata of the resource.

To this end it can be helpful to check the logs displayed at https://bioimageio-uploader.netlify.app/#/status/<concept_id>. There information about automated workflow steps and validation outcome is logged. Additionally an 'error' status may be shown if an exception occured in the GitHub workflow run producing the log. To be able to see all the packaged models along with the 'rdf.yaml' as well as covers and other documentation, go to the draft collection and download the model from there.

For models, reviewers can use the model documentation as a guide.

Reviewers can:

• request changes: A contributor is expected to upload an updated (fixed) draft (which overwrites the current draft). This can be done either by the website, or the stage workflow. Important is to keep the id the same.
• apply minor changes: For minor changes it is often not necessary to involve the contributor directly. Such changes could be e.g. obvious typos in some of the fields. Reviewers can apply changes locally to the resource package and re-upload (using the same id), or, given S3 credentials, files could also be changed in place. Latter way of changing requires manually triggering the tests (see Testing section).
• accept: Accepting the resource draft via the web interface triggers the publish workflow, which creates a new unique resource version. As a result, the resource is published, the draft deleted and, thus, the resource is available via the bioimage.io website. The backup workflow will upload/publish the resource version to zenodo using the bioimage.io bot account (tagged with backup.bioimage.io).

Add community partner

To link yourself as a community partner, please create a PR to insert relevant metadata into bioimageio_collection_config.json under "partners". Checkout ilastik partner entry for an example.

Add community partner compatibility checks

Any community partner is invited to add a GitHub Actions workflow in this repo (please make a PR) that generates reports on its software compatibility with new and updated resources in the bioimage.io collection. See ilastik compatibility checks workflow for an example.

If you are not familiar with GitHub Actions workflows, we can help you to set this up analog to our existing community partner compatibility checks. Ideally you can provide a script to create a compatibility report (a relativley simple json file) for a given resource description. see this Python script as an example.

Footnotes

1. Parameters to this workflow are bioimage.io resource concept ID (id from the rdf.yaml), and Download URL of the resource package zip-file, which should contain a publicly reachable URL to a resource package .zip-file. ↩

2. Parameters to this workflow are Bioimage.io resource concept ID (id from the rdf.yaml), and Published version or 'draft' (optional, usually draft). ↩