Widget to transform cells to an atlas space #117

dstansby · 2022-05-19T15:12:24Z

Fixes brainglobe/cellfinder#316, providing a plugin for users to transform their detected cells to an atlas space, where the image that they detected cells in has previously been registeterd by brainreg.

deprecated-napari-hub-preview-bot · 2022-05-19T15:17:29Z

Preview page for your plugin is ready here:
https://preview.napari-hub.org/brainglobe/cellfinder-napari/117
Updated: 2022-06-29T13:50:17.059457

codecov-commenter · 2022-06-10T12:48:21Z

Codecov Report

Merging #117 (dcb3502) into main (d5c94b6) will increase coverage by 13.88%.
The diff coverage is 91.93%.

@@             Coverage Diff             @@
##             main     brainglobe/cellfinder#117       +/-   ##
===========================================
+ Coverage   81.28%   95.17%   +13.88%     
===========================================
  Files          15       19        +4     
  Lines         668      870      +202     
===========================================
+ Hits          543      828      +285     
+ Misses        125       42       -83

Impacted Files	Coverage Δ
cellfinder_napari/transform.py	`85.71% <85.71%> (ø)`
cellfinder_napari/tests/test_transform.py	`100.00% <100.00%> (ø)`
cellfinder_napari/tests/test_curation.py	`100.00% <0.00%> (ø)`
cellfinder_napari/tests/test_training.py	`100.00% <0.00%> (ø)`
cellfinder_napari/tests/test_detection.py	`100.00% <0.00%> (ø)`
cellfinder_napari/tests/test_train_containers.py	`100.00% <0.00%> (ø)`
cellfinder_napari/tests/test_utils.py	`100.00% <0.00%> (ø)`
cellfinder_napari/curation.py	`89.83% <0.00%> (+6.77%)`	⬆️
cellfinder_napari/train/train_containers.py	`100.00% <0.00%> (+20.96%)`	⬆️
cellfinder_napari/utils.py	`93.54% <0.00%> (+29.03%)`	⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d5c94b6...dcb3502. Read the comment docs.

Add feedback if not all layers selected Working transformation widget

Open new plugin in example Cast gaussian to int [delete] add registered image layers Update example

paddyroddy

A few concerns/comments on Git LFS

cellfinder_napari/tests/test_transform.py

paddyroddy · 2022-06-30T09:12:20Z

cellfinder_napari/transform.py

+    if _DEFORMATION_FIELD_DIRECTORY is None:
+        reg_dir = Path(reg_meta["registration_output_folder"])
+    else:
+        reg_dir = _DEFORMATION_FIELD_DIRECTORY


Suggested change

if _DEFORMATION_FIELD_DIRECTORY is None:

reg_dir = Path(reg_meta["registration_output_folder"])

else:

reg_dir = _DEFORMATION_FIELD_DIRECTORY

reg_dir = Path(reg_meta["registration_output_folder"]) if _DEFORMATION_FIELD_DIRECTORY is None else _DEFORMATION_FIELD_DIRECTORY

paddyroddy · 2022-06-30T09:16:40Z

cellfinder_napari/transform.py

+    else:
+        reg_dir = _DEFORMATION_FIELD_DIRECTORY
+    deformation_field_paths = [
+        str(reg_dir / f"deformation_field_{i}.tiff") for i in [0, 1, 2]


I see that ultimately this is passed to tifflike.imread. Does it not accept Path objects?

paddyroddy · 2022-06-30T09:29:34Z

.github/workflows/test_and_deploy.yml

@@ -34,6 +34,9 @@ jobs:
          python-version: "3.9"

    steps:
+      - uses: actions/checkout@v3
+        with:
+          lfs: true


The actions/checkout docs are vague. I was wondering if one could specify the files manually instead of just everything tracked by Git LFS - in case we don't need all of them for the CI.

See this issue regarding my concerns on bandwidth actions/checkout#834

Thanks for picking up on this, I wasn't aware of that issue 😃 We should probably upload the test data to a server designed for data sharing then. @adamltyson since I remember using https://gin.g-node.org in a previous PR, perhaps we should set up a new repository there for test data?

Yeah I'd rather we continued to host all large files at https://gin.g-node.org/BrainGlobe.

paddyroddy · 2022-06-30T09:31:40Z

.gitattributes

@@ -0,0 +1 @@
+*.tiff filter=lfs diff=lfs merge=lfs -text


Just wondering whether a catch-all *.tiff is risky here? I think that we should be more explicit either here or in the .gitignore. Not sure if you've encountered problems with Git LFS before, but I have, and it can very quickly consume all your free quota - I've never paid for it though.

adamltyson · 2022-07-21T10:45:04Z

Need to look into this further. Although the examples run fine, I get errors with "real" data, e.g.:

IndexError: index 356 is out of bounds for axis 1 with size 310

adamltyson · 2023-03-23T11:09:04Z

@alessandrofelder would you be able to take a look at this sometime? It's been open for nearly a year, but I'd still like to get it merged.

The issues I remember are:

It fails on the real data workflows I tested it on
I'd like to avoid using git lfs if possible, and keep all our hosted together in one place (currently GIN)

alessandrofelder · 2023-03-31T14:04:36Z

cellfinder_napari/tests/data/brainreg-napari-output/brainreg.json

@@ -0,0 +1 @@
+{"image_paths": null, "backend": "niftyreg", "voxel_sizes": ["50.0", "40.0", "40.0"], "orientation": "psl", "data_orientation": "psl", "atlas": "example_mouse_100um", "atlas_key": "atlas_key.example_mouse_100um", "registration_output_folder": "/Users/dstansby/brainreg-napari-output", "affine_n_steps": "6", "affine_use_n_steps": "5", "freeform_n_steps": "6", "freeform_use_n_steps": "4", "bending_energy_weight": "0.95", "grid_spacing": "10", "smoothing_sigma_reference": "1", "smoothing_sigma_floating": "1.0", "histogram_n_bins_floating": "128.0", "histogram_n_bins_reference": "128.0"}


Locally tests fail the assertion on Line 57 of test_transform.py

Manually emulating the test in the GUI gives:

Traceback (most recent call last): File "src\psygnal\_signal.py", line 950, in _run_emit_loop File "src\psygnal\_weak_callback.py", line 262, in cb File "C:\Users\Alessandro\anaconda3\envs\brainglobe\lib\site-packages\magicgui\widgets\_function_gui.py", line 218, in _disable_button_and_call self.__call__() File "C:\Users\Alessandro\anaconda3\envs\brainglobe\lib\site-packages\magicgui\widgets\_function_gui.py", line 331, in __call__ value = self._function(*bound.args, **bound.kwargs) File "C:\Users\Alessandro\Documents\UCL-projects\napari-plugins\cellfinder-napari\cellfinder_napari\transform.py", line 72, in transform transformed_cells = transform_points_to_atlas_space( File "C:\Users\Alessandro\anaconda3\envs\brainglobe\lib\site-packages\cellfinder\analyse\analyse.py", line 230, in transform_points_to_atlas_space return transform_points_downsampled_to_atlas_space( File "C:\Users\Alessandro\anaconda3\envs\brainglobe\lib\site-packages\cellfinder\analyse\analyse.py", line 265, in transform_points_downsampled_to_atlas_space deformation_field = tifffile.imread(deformation_field_path) File "C:\Users\Alessandro\anaconda3\envs\brainglobe\lib\site-packages\tifffile\tifffile.py", line 1040, in imread with TiffFile( File "C:\Users\Alessandro\anaconda3\envs\brainglobe\lib\site-packages\tifffile\tifffile.py", line 3933, in __init__ fh = FileHandle(file, mode=mode, name=name, offset=offset, size=size) File "C:\Users\Alessandro\anaconda3\envs\brainglobe\lib\site-packages\tifffile\tifffile.py", line 13635, in __init__ self.open() File "C:\Users\Alessandro\anaconda3\envs\brainglobe\lib\site-packages\tifffile\tifffile.py", line 13650, in open self._fh = open(self._file, self._mode) # type: ignore FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\dstansby\\brainreg-napari-output\\deformation_field_0.tiff' The above exception was the direct cause of the following exception: Traceback (most recent call last): File "C:\Users\Alessandro\anaconda3\envs\brainglobe\lib\site-packages\magicgui\widgets\bases\_value_widget.py", line 65, in _on_value_change self.changed.emit(value) File "src\psygnal\_signal.py", line 905, in emit File "src\psygnal\_signal.py", line 952, in _run_emit_loop psygnal._signal.EmitLoopError: calling <psygnal._weak_callback._StrongFunction object at 0x0000025851D9B5C0> with args=(False,) caused FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\dstansby\\brainreg-napari-output\\deformation_field_0.tiff'.

I suspect the hard-coded path here has something to do with it...
{" "registration_output_folder": "/Users/dstansby/brainreg-napari-output"}
... but how did the CI pass all that time ago? 🤔

alessandrofelder · 2023-05-19T13:20:06Z

The error we encountered here is a consequence of brainglobe/brainglobe-workflows#42, where it should actually be fixed.

In discussion with @adamltyson we decided to discontinue the attempt at making a transform widget because

as it's not high-priority enough
what we want napari users to be able to do in this context is likely to change soon.

Therefore, closing and closing brainglobe/cellfinder#153.

dstansby force-pushed the transform-widget branch from 6e4bfbf to 3443615 Compare June 10, 2022 12:11

dstansby mentioned this pull request Jun 24, 2022

Make sure metadata is attached to loaded napari Layers brainglobe/brainreg-napari#38

Merged

4 tasks

dstansby added 3 commits June 27, 2022 16:11

Start widget to transform cells to registered atlas space

bf36f44

Add feedback if not all layers selected Working transformation widget

Example to add fake cells to whole brain

d6a46ab

Open new plugin in example Cast gaussian to int [delete] add registered image layers Update example

Add a test

1fd77c5

dstansby force-pushed the transform-widget branch from 3443615 to 1fd77c5 Compare June 27, 2022 15:11

Add brainreg-napari to requirements

05f4b6f

dstansby force-pushed the transform-widget branch from f858511 to 05f4b6f Compare June 29, 2022 10:58

Add cellfinder to requirements

ea827c6

dstansby force-pushed the transform-widget branch from d5fcceb to ea827c6 Compare June 29, 2022 11:09

dstansby added 2 commits June 29, 2022 13:06

Track tiff files with lfs

6d4c2a4

Add test data to test transformation

a6f8869

dstansby force-pushed the transform-widget branch from 8c5e1c7 to a6f8869 Compare June 29, 2022 12:35

dstansby added 2 commits June 29, 2022 13:59

Checkout repository with lfs=true

83ac187

Set deformation file paths at start of testing

dcb3502

dstansby force-pushed the transform-widget branch from 158ba3b to dcb3502 Compare June 29, 2022 13:46

dstansby marked this pull request as ready for review June 29, 2022 15:15

dstansby requested review from adamltyson and a team June 29, 2022 15:52

paddyroddy approved these changes Jun 30, 2022

View reviewed changes

alessandrofelder removed the request for review from adamltyson March 31, 2023 13:53

alessandrofelder reviewed Mar 31, 2023

View reviewed changes

alessandrofelder mentioned this pull request Apr 5, 2023

Transform widget (2nd iteration) #153

Closed

12 tasks

adamltyson mentioned this pull request May 19, 2023

indexing error, analysis step fails brainglobe/brainglobe-workflows#42

Closed

alessandrofelder closed this May 19, 2023

adamltyson mentioned this pull request Jan 12, 2024

Add "Layer interpreters" as independent functionality and use them in BrainGlobe brainglobe/BrainGlobe#10

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Widget to transform cells to an atlas space #117

Widget to transform cells to an atlas space #117

dstansby commented May 19, 2022 •

edited

Loading

deprecated-napari-hub-preview-bot bot commented May 19, 2022 •

edited

Loading

codecov-commenter commented Jun 10, 2022 •

edited

Loading

paddyroddy left a comment

paddyroddy Jun 30, 2022

paddyroddy Jun 30, 2022

paddyroddy Jun 30, 2022 •

edited

Loading

dstansby Jun 30, 2022

adamltyson Jul 1, 2022

paddyroddy Jun 30, 2022

adamltyson commented Jul 21, 2022

adamltyson commented Mar 23, 2023

alessandrofelder Mar 31, 2023

alessandrofelder commented May 19, 2023

		@@ -0,0 +1 @@
		{"image_paths": null, "backend": "niftyreg", "voxel_sizes": ["50.0", "40.0", "40.0"], "orientation": "psl", "data_orientation": "psl", "atlas": "example_mouse_100um", "atlas_key": "atlas_key.example_mouse_100um", "registration_output_folder": "/Users/dstansby/brainreg-napari-output", "affine_n_steps": "6", "affine_use_n_steps": "5", "freeform_n_steps": "6", "freeform_use_n_steps": "4", "bending_energy_weight": "0.95", "grid_spacing": "10", "smoothing_sigma_reference": "1", "smoothing_sigma_floating": "1.0", "histogram_n_bins_floating": "128.0", "histogram_n_bins_reference": "128.0"}

Widget to transform cells to an atlas space #117

Widget to transform cells to an atlas space #117

Conversation

dstansby commented May 19, 2022 • edited Loading

deprecated-napari-hub-preview-bot bot commented May 19, 2022 • edited Loading

codecov-commenter commented Jun 10, 2022 • edited Loading

Codecov Report

paddyroddy left a comment

Choose a reason for hiding this comment

paddyroddy Jun 30, 2022

Choose a reason for hiding this comment

paddyroddy Jun 30, 2022

Choose a reason for hiding this comment

paddyroddy Jun 30, 2022 • edited Loading

Choose a reason for hiding this comment

dstansby Jun 30, 2022

Choose a reason for hiding this comment

adamltyson Jul 1, 2022

Choose a reason for hiding this comment

paddyroddy Jun 30, 2022

Choose a reason for hiding this comment

adamltyson commented Jul 21, 2022

adamltyson commented Mar 23, 2023

alessandrofelder Mar 31, 2023

Choose a reason for hiding this comment

alessandrofelder commented May 19, 2023

dstansby commented May 19, 2022 •

edited

Loading

deprecated-napari-hub-preview-bot bot commented May 19, 2022 •

edited

Loading

codecov-commenter commented Jun 10, 2022 •

edited

Loading

paddyroddy Jun 30, 2022 •

edited

Loading