Annotations don't load/render when Slim viewer backed by DICOMWeb service of Google Cloud Healthcare API. Affects Pan-Cancer-Nuceli-Seg-DICOM collection in IDC Portal.

[helmut-hoffer-von-ankershoffen]

TL;DR Loading/rendering of annotations broken when Slim viewer backed by DICOMWeb service of Google Cloud Healthcare API. This affects the IDC portal, in particular the Pan-Cancer-Nuceli-Seg-DICOM collection. This affects Google Cloud deployments set up according to the installation instructions provided by Slim viewer. When Slim viewer is backed by dcm4che, loading/rendering of annotations works, for the same study / dicom files.

This is imo critical.

Steps to reproduce

Case 1: IDC Portal -> fails

1. Go to to Pan-Cancer-Nuceli-Seg-DICOM collection on IDC Portal,
2. Select any study in this collection
3. Click on View for Selected Study, which opens Slim Viewer.
4. Navigate to slide with annotations (ending with -DX1), uncollapse Annotation Groups
5. Enable "Nuclei" annotation group
   Result: Various error messages displayed and shown in browser console. No annotations rendered.

Case 2: Separate Google Cloud deployment -> fails

1. Follow installation instructions Google Cloud deployment of the Slim Viewer
2. Download study of Pan-Cancer-Nuclei-Seg-DICOM, e.g. idc download 2.25.234091009181871968405618923996208732870
3. Store study in DICOMWeb installation created in step Allow forking? #1
4. Open the study in Slim Viewer, enable Nuclei annotation group
   Result: Various error messages displayed and shown in browser console. No annotations rendered.

Case 3: Local installation of dcm4che -> works

1. Run local dcm4che installation using docker-compose.yml provided by Slim viewer
2. Store study referenced above in local dcm4che installation
3. Open the study in Slim Viewer, enable Nuclei annotation group
   Results: Clusters and when zooming in annotations are shown, as expected

[helmut-hoffer-von-ankershoffen]

FYI @fedorov.

Annotations with Google Healthcare API now working.

Resolution: the simple "fix" is to use the preview version of the Google Healthcare API, i.e. use https://healthcare.googleapis.com/v1beta1/... in the config / servers.url.

I kindly suggest you (1) add this to your documentation in a prominent place, and (2) evaluate to apply for the IDC Portal of NCI, as rendering of all bulk annotations is broken there.

Note: Might be worth a try to check if that as well resolves #169

[fedorov]

@helmut-hoffer-von-ankershoffen glad you figured it out!

While we did release this dataset in IDC (sharp eyes!), we intentionally did not advertise it exactly because we have not completely sorted out the visualization issues in Slim. There is some groundwork in the IDC proxy that needs to be done first to properly handle bulk data requests (we are working on this), but also on the viewer side, we sometime run into memory issues with large annotation sizes. We are also planning to implement download in a separate thread so the UI is not locked.

#169 is a separate issue, and it was not working due to missing support of BitsAllocated=1 as needed for DICOM SEG. That was implemented in production Google Healthcare API, and we confirmed that is resolved.

I kindly suggest you (1) add this to your documentation in a prominent place, and (2) evaluate to apply for the IDC Portal of NCI, as rendering of all bulk annotations is broken there.

We will, once all the functionality we are working on is in place.

But since you discovered this dataset, can you share any comments about using those bulk annotations with Slim, especially large ones? Do you have any comments or wishes?

[helmut-hoffer-von-ankershoffen]

Hi @fedorov ,

thx for confirming.

1. Interesting you mention a proxy. Do I assume right that proxy sits between Google's DICOMWeb service and Slim Viewer, specifically for he IDC portal of cancer.gov? What is the functionality of that proxy? For note: I was actually tinkering with the idea to establish a proxy in the architectural exploration of DICOM(web) I am doing right now, to bring in aspects I see missing in the standards, e.g. a) slide level authorization, b) spatial retrieval of bulk annotations (i.e. caching and querying of annotations matching tiles/frames rendered), and c) further extensions required for our use cases.
2. Thx for asking for further comments/wishes: I will explore DICOMweb, Slim Viewer and limitations a bit more, and circle back with additional observations and a wish list in a few weeks.
3. Reg. not advertising the presence of the dataset in the IDC portal - It's actually mentioned in this nature communication ;-)

[fedorov]

Interesting you mention a proxy. Do I assume right that proxy sits between Google's DICOMWeb service and Slim Viewer, specifically for he IDC portal of cancer.gov? What is the functionality of that proxy?

We talk briefly about the proxy here: https://learn.canceridc.dev/portal/proxy-policy. The main reasons for the proxy:

1. we provide open, un-authenticated visualization of IDC images. It is currently - unfortunately - impossible to have a publicly available Google DICOM store.
2. data egress from the store is not free, and we need to have the means to control costs, which is why we have quota in place, as mentioned in the article above.

Aside from taking care of the 2 points above, proxy is passing everything through, with the exception of bulk data requests, which is currently under development. Currently, proxy points to "v1", not "v1beta1" API.

Reg. not advertising the presence of the dataset in the IDC portal - It's actually mentioned in this nature communication ;-)

Nice! I forgot we mentioned it there. That was an aspirational statement, as stated in the paper: "The DICOM Microscopy Bulk Simple Annotations and Segmentation objects are in the process of being included into the IDC and will soon become available via the IDC portal, too." "Soon" became reality just a few month ago (and binary segmentations are still not in!)! :-D Anyway, no need to hide that those annotations are in IDC - it's just that we are not highlighting its availability until users can visualize them in the viewer.