September 27, 2022 (Dev Meeting)

Agenda

#	Owner	Title	Time (min)
1	Jonny	MONAI Label workflow for pathology	0
2	Sachi	DSA interactivity and ROI selection	0
3	Behrooz	Active learning for pathology	0

Attendees

Behrooz Hashemian, David Manthey, Gregory Lee, Lee Cooper, Jeff Baumes, Jonny Hancox, Michael Boone, Ziyue Xu, Gigon Bae, Raza, Sachidanand Alle, Andres Diaz-Pinto

Notes

• We discussed about the workflow for pathologist and data scientists interacting with MONAI Label. Jonny presented the proposed workflow:

  1. Pathologist who wants to analyse cell subtypes
     • Opens Viewer (assuming Monai Label (ML) Plugin is already installed)
     • User selects WSI image source for analysis
     • User selects region of interest and chooses ML tool to run inference on the region
     • ML returns the centroids, colour-coded by subtype and displays counts for each sub-type
     • User can either multi-select false positives or incorrectly located instances and then delete them or single-click delete individual instances.
     • User can add false negatives with single click, by subtype
     • For any remaining editing, they could use Viewer native tools
     • When they have completed the ROI annotation they click the Submit Annotation option.
  • Notes:
    • Users can, optionally, filter the view by cell sub-type
    • We can still use Hovernet here, the post-processing it includes can return centroids and nucleus counts
    • It would be good if the system could provide an interpolation option, so that as they annotate ROIs on a WSI, it estimates how may there are per-subtype on the whole WSI
    • When adding missing cells with single click, the system could, optionally offer a snap to estimated centroid feature, in which it uses something like Nuclick in the background to segment where the user clicked and then accurately get the centroid, but not sure whether there is any value. It depends whether it is faster for the pathologist that doing it themselves.
    • Users may prefer to annotate by subtype. This could provide useful conditioning for inference if they select a region and specify one or more subtypes that are present in that region before running the prediction.
  2. Data Scientist who wants to train/fine tune a nucleus classification and segmentation model.
     • Opens Viewer (assuming ML Plugin is already installed)
     • Opens ML Tool
     • User selects image source for training (directory or single WSI)
     • Concurrently, ML Server runs through the images, thresholds them, runs inference and builds up a list ordered by patch uncertainty
     • User chooses Next patch option
     • ML sends highest value patch (with generous border for context) to the Viewer and displays the inference result (pretty much as it does now for an ROI), with colour coding for sub-types
     • User can either multi-select false positives or incorrectly segmented instances and then delete them or optionally, use toggle a tool that allows single-click deletion of individual instances.
     • For nucleus instances that have been merged during segmentation, there could be a tool that allows the user to click the centroids of each distinct nucleus within the merged region. Watershed or similar could then be used to divide the merged region
     • User selects Nuclick tool from ML tools to define any false negatives or redefine any deleted instances
     • For any remaining editing, they could use Viewer's native tools
     • When they have completed the patch annotation, they click the ML Next Patch option, which automatically submits their annotations and retrains offline.
  • Notes:
    • This workflow mirrors the current radiology use-case for ML
    • Users can define the dimensions of patches, it should default to a region containing a number of nuclei that they can reasonably edit/reannotate in a couple of minutes.
    • Users should, optionally, be able to filter the view by cell sub-type when editing
    • In this use-case, with QuPath as the viewer, it only needs to load the ROI being annotated. Perhaps the plug-in could create a temp file locally of the ROI and use the API to load it.

• Tumor Infiltrating Lymphocytes was proposed as a use case. Jonny mentioned that although it is easy to identify, scoring is challenging. Lee also showed the classes of immune-flamed, immune-excluded, immune-desert tumors.

• There was consensus on importance and usefulness of active learning for digital pathology. For the our use cases we need he interaction model for nuclear segmentation (NuClick). Sachi mentioned that selecting the right patches within one slide is more tractable than selecting the right WSI. David mentioned that DSA has an efficient way to switch between WSI.

• Sachi discussed DSA interactivity issues with David. Interactivity needs high-speed transaction of data between MONAI Label and DSA but since the transmitted data are in JSON format, decoding and encoding it is not efficient. David will look into it to use other data formats like pickle.

Action items

PIC	Item
David	To examine the way to enable interactive labeling for MONAI Label + DSA
Sachi	To sync with Daguang, Vish, Ziyue on active learning for pathology