Project Dataset Image Folders

[will-moore]

"More levels" has been a feature request for a long time.
More recent examples: https://www.openmicroscopy.org/community/viewtopic.php?f=4&t=8419
Also feedback from TIM 2018 meeting and IDR mailing lists.

Several design decisions needed to consider:

• UI
• Backend implementation
• Upgrade from current situation

Folder structure

• Do we need/want many-to-many relationships, or should each Folder have just a single parent?
  • Many-to-many is a fair bit more work in the client and with multiple levels could get kinda confusing
• Do we simply add more levels on top of Screens/Projects, or replace Screens/Projects with Folders. Could even get rid of Datasets and just put Images into Folders?
• Keeping Screens, Projects and Datasets, allowing users to add Screens/Projects to Folders is the least intrusive for upgrading since all existing hierarchies are kept. However, having more types of container is confusing: Folder, Project, Dataset, Screen and Plate. Could just have Folder & Plate.
• How do we distinguish between Folders that are Image/Plate containers and others (e.g. ROI Folders)? Also, need to know which are the top-level (orphaned) Folders to load first.
• Path_to_object logic needs to be able to traverse this hierarchy.

To start assessing some options, I began a prototype in the webclient, https://github.com/will-moore/openmicroscopy/tree/project_folders
simply using Tags with a 'project.folder' namespace as a backend for representing folders. These contain Projects only in the current implementation.

Already we can see this is probably a bit confusing with all the various containers in the tree (this is with omero.client.ui.tree.type_order=False so all containers are mixed together).
So probably a good time to consider some of the options above.

cc @pwalczysko @jburel @chris-allan @joshmoore @sbesson

[joshmoore]

Some initial thoughts:

Do we need/want many-to-many relationships, or should each Folder have just a single parent?

Folders as they currently stand are one-to-many, but the contents can be in multiple folders.

Do we simply add more levels on top of Screens/Projects,...just put Images into Folders

If it's "just put the import item" (i.e. images or plates) in folders, then I could see that.

How do we distinguish between Folders that are Image/Plate containers and others (e.g. ROI Folders)?

Annotation folders come quickly to mind. At the moment, they are strongly typed so it shouldn't be difficult to differentiate if we don't change the strategy:

• RoiFolders
• AnnotationFolders
• ImportFolders

[will-moore]

OK, so if we get rid of Projects and Datasets then that'll happen at upgrade time, with a script that converts Projects and Datasets and Screens to AnnotationFolders.
If AnnotationFolders only have a single parent, we'd have to handle Datasets in multiple Projects somehow?
Also, getting rid of Datasets mean things like OMERO.scripts, iviewer etc have to handle Folders instead of Datasets.

[joshmoore]

with a script that converts Projects and Datasets and Screens to AnnotationFolders.

To ImportFolders? (or whatever they're called)

Also, getting rid of Datasets mean things like OMERO.scripts, iviewer etc have to handle Folders instead of Datasets.

Yeah, there's going to be a long list of things like this. If keeping Datasets keeps most things simple, then I could imagine keeping them, but that will increase the long-term complexity.

[mtbc]

Folders at present can contain a mix of ROIs and images. I think this heterogeneity may be a feature allowing users to organize objects by what they relate to rather than what they are. It may depend in part on how we someday represent correlative imaging, etc. Tightly typed folders are certainly an option though.

[jburel]

Such feature was again mention during the latest training session in Singapore