Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

The evaluation results using the provided pretrained checkpoint are not consistent with that in the paper. #14

Open
ryylcc opened this issue Aug 15, 2023 · 15 comments
Open

The evaluation results using the provided pretrained checkpoint are not consistent with that in the paper. #14

ryylcc opened this issue Aug 15, 2023 · 15 comments

Comments

@ryylcc
Copy link

ryylcc commented Aug 15, 2023

No description provided.

@ryylcc ryylcc changed the title The evaluation results of the 3DSAM-adapter is The evaluation results of the 3DSAM-adapter is not consistent with that in the Aug 15, 2023
@ryylcc ryylcc changed the title The evaluation results of the 3DSAM-adapter is not consistent with that in the The evaluation results using the provided pretrained checkpoint are not consistent with that in the paper. Aug 15, 2023
@ryylcc
Copy link
Author

ryylcc commented Aug 15, 2023 


[14:46:55.321]  Case ./data/Task10_Colon/imagesTr/colon_008.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:46:55,321 -  Case ./data/Task10_Colon/imagesTr/colon_008.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:01.455]  Case ./data/Task10_Colon/imagesTr/colon_009.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:01,455 -  Case ./data/Task10_Colon/imagesTr/colon_009.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:07.196]  Case ./data/Task10_Colon/imagesTr/colon_026.nii.gz - Dice 0.015407 | NSD 0.065295
2023-08-15 14:47:07,196 -  Case ./data/Task10_Colon/imagesTr/colon_026.nii.gz - Dice 0.015407 | NSD 0.065295
[14:47:12.848]  Case ./data/Task10_Colon/imagesTr/colon_030.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:12,848 -  Case ./data/Task10_Colon/imagesTr/colon_030.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:20.227]  Case ./data/Task10_Colon/imagesTr/colon_040.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:20,227 -  Case ./data/Task10_Colon/imagesTr/colon_040.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:30.019]  Case ./data/Task10_Colon/imagesTr/colon_046.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:30,019 -  Case ./data/Task10_Colon/imagesTr/colon_046.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:39.798]  Case ./data/Task10_Colon/imagesTr/colon_091.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:39,798 -  Case ./data/Task10_Colon/imagesTr/colon_091.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:46.366]  Case ./data/Task10_Colon/imagesTr/colon_095.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:46,366 -  Case ./data/Task10_Colon/imagesTr/colon_095.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:53.232]  Case ./data/Task10_Colon/imagesTr/colon_098.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:53,232 -  Case ./data/Task10_Colon/imagesTr/colon_098.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:59.353]  Case ./data/Task10_Colon/imagesTr/colon_104.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:59,353 -  Case ./data/Task10_Colon/imagesTr/colon_104.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:07.321]  Case ./data/Task10_Colon/imagesTr/colon_129.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:07,321 -  Case ./data/Task10_Colon/imagesTr/colon_129.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:15.462]  Case ./data/Task10_Colon/imagesTr/colon_133.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:15,462 -  Case ./data/Task10_Colon/imagesTr/colon_133.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:25.344]  Case ./data/Task10_Colon/imagesTr/colon_136.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:25,344 -  Case ./data/Task10_Colon/imagesTr/colon_136.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:32.039]  Case ./data/Task10_Colon/imagesTr/colon_141.nii.gz - Dice 0.000000 | NSD 0.017542
2023-08-15 14:48:32,039 -  Case ./data/Task10_Colon/imagesTr/colon_141.nii.gz - Dice 0.000000 | NSD 0.017542
[14:48:38.911]  Case ./data/Task10_Colon/imagesTr/colon_142.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:38,911 -  Case ./data/Task10_Colon/imagesTr/colon_142.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:44.344]  Case ./data/Task10_Colon/imagesTr/colon_145.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:44,344 -  Case ./data/Task10_Colon/imagesTr/colon_145.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:53.632]  Case ./data/Task10_Colon/imagesTr/colon_163.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:53,632 -  Case ./data/Task10_Colon/imagesTr/colon_163.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:00.167]  Case ./data/Task10_Colon/imagesTr/colon_164.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:00,167 -  Case ./data/Task10_Colon/imagesTr/colon_164.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:06.922]  Case ./data/Task10_Colon/imagesTr/colon_181.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:06,922 -  Case ./data/Task10_Colon/imagesTr/colon_181.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:14.982]  Case ./data/Task10_Colon/imagesTr/colon_194.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:14,982 -  Case ./data/Task10_Colon/imagesTr/colon_194.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:21.982]  Case ./data/Task10_Colon/imagesTr/colon_202.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:21,982 -  Case ./data/Task10_Colon/imagesTr/colon_202.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:26.959]  Case ./data/Task10_Colon/imagesTr/colon_205.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:26,959 -  Case ./data/Task10_Colon/imagesTr/colon_205.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:33.510]  Case ./data/Task10_Colon/imagesTr/colon_207.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:33,510 -  Case ./data/Task10_Colon/imagesTr/colon_207.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:41.245]  Case ./data/Task10_Colon/imagesTr/colon_208.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:41,245 -  Case ./data/Task10_Colon/imagesTr/colon_208.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:50.006]  Case ./data/Task10_Colon/imagesTr/colon_215.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:50,006 -  Case ./data/Task10_Colon/imagesTr/colon_215.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:57.969]  Case ./data/Task10_Colon/imagesTr/colon_218.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:57,969 -  Case ./data/Task10_Colon/imagesTr/colon_218.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:58,035 - - Test metrics Dice: 0.00059259165
2023-08-15 14:49:58,035 - - Test metrics NSD: 0.0031860285794686172

@peterant330
Copy link
Collaborator 


[14:46:55.321]  Case ./data/Task10_Colon/imagesTr/colon_008.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:46:55,321 -  Case ./data/Task10_Colon/imagesTr/colon_008.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:01.455]  Case ./data/Task10_Colon/imagesTr/colon_009.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:01,455 -  Case ./data/Task10_Colon/imagesTr/colon_009.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:07.196]  Case ./data/Task10_Colon/imagesTr/colon_026.nii.gz - Dice 0.015407 | NSD 0.065295
2023-08-15 14:47:07,196 -  Case ./data/Task10_Colon/imagesTr/colon_026.nii.gz - Dice 0.015407 | NSD 0.065295
[14:47:12.848]  Case ./data/Task10_Colon/imagesTr/colon_030.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:12,848 -  Case ./data/Task10_Colon/imagesTr/colon_030.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:20.227]  Case ./data/Task10_Colon/imagesTr/colon_040.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:20,227 -  Case ./data/Task10_Colon/imagesTr/colon_040.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:30.019]  Case ./data/Task10_Colon/imagesTr/colon_046.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:30,019 -  Case ./data/Task10_Colon/imagesTr/colon_046.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:39.798]  Case ./data/Task10_Colon/imagesTr/colon_091.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:39,798 -  Case ./data/Task10_Colon/imagesTr/colon_091.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:46.366]  Case ./data/Task10_Colon/imagesTr/colon_095.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:46,366 -  Case ./data/Task10_Colon/imagesTr/colon_095.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:53.232]  Case ./data/Task10_Colon/imagesTr/colon_098.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:53,232 -  Case ./data/Task10_Colon/imagesTr/colon_098.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:59.353]  Case ./data/Task10_Colon/imagesTr/colon_104.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:59,353 -  Case ./data/Task10_Colon/imagesTr/colon_104.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:07.321]  Case ./data/Task10_Colon/imagesTr/colon_129.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:07,321 -  Case ./data/Task10_Colon/imagesTr/colon_129.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:15.462]  Case ./data/Task10_Colon/imagesTr/colon_133.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:15,462 -  Case ./data/Task10_Colon/imagesTr/colon_133.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:25.344]  Case ./data/Task10_Colon/imagesTr/colon_136.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:25,344 -  Case ./data/Task10_Colon/imagesTr/colon_136.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:32.039]  Case ./data/Task10_Colon/imagesTr/colon_141.nii.gz - Dice 0.000000 | NSD 0.017542
2023-08-15 14:48:32,039 -  Case ./data/Task10_Colon/imagesTr/colon_141.nii.gz - Dice 0.000000 | NSD 0.017542
[14:48:38.911]  Case ./data/Task10_Colon/imagesTr/colon_142.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:38,911 -  Case ./data/Task10_Colon/imagesTr/colon_142.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:44.344]  Case ./data/Task10_Colon/imagesTr/colon_145.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:44,344 -  Case ./data/Task10_Colon/imagesTr/colon_145.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:53.632]  Case ./data/Task10_Colon/imagesTr/colon_163.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:53,632 -  Case ./data/Task10_Colon/imagesTr/colon_163.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:00.167]  Case ./data/Task10_Colon/imagesTr/colon_164.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:00,167 -  Case ./data/Task10_Colon/imagesTr/colon_164.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:06.922]  Case ./data/Task10_Colon/imagesTr/colon_181.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:06,922 -  Case ./data/Task10_Colon/imagesTr/colon_181.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:14.982]  Case ./data/Task10_Colon/imagesTr/colon_194.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:14,982 -  Case ./data/Task10_Colon/imagesTr/colon_194.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:21.982]  Case ./data/Task10_Colon/imagesTr/colon_202.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:21,982 -  Case ./data/Task10_Colon/imagesTr/colon_202.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:26.959]  Case ./data/Task10_Colon/imagesTr/colon_205.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:26,959 -  Case ./data/Task10_Colon/imagesTr/colon_205.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:33.510]  Case ./data/Task10_Colon/imagesTr/colon_207.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:33,510 -  Case ./data/Task10_Colon/imagesTr/colon_207.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:41.245]  Case ./data/Task10_Colon/imagesTr/colon_208.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:41,245 -  Case ./data/Task10_Colon/imagesTr/colon_208.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:50.006]  Case ./data/Task10_Colon/imagesTr/colon_215.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:50,006 -  Case ./data/Task10_Colon/imagesTr/colon_215.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:57.969]  Case ./data/Task10_Colon/imagesTr/colon_218.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:57,969 -  Case ./data/Task10_Colon/imagesTr/colon_218.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:58,035 - - Test metrics Dice: 0.00059259165
2023-08-15 14:49:58,035 - - Test metrics NSD: 0.0031860285794686172

Sorry we just did some modification to the training and testing code, but the checkpoint is not updated yet. I will update the checkpoint within this week.

@CunminZhao
Copy link 


[14:46:55.321]  Case ./data/Task10_Colon/imagesTr/colon_008.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:46:55,321 -  Case ./data/Task10_Colon/imagesTr/colon_008.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:01.455]  Case ./data/Task10_Colon/imagesTr/colon_009.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:01,455 -  Case ./data/Task10_Colon/imagesTr/colon_009.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:07.196]  Case ./data/Task10_Colon/imagesTr/colon_026.nii.gz - Dice 0.015407 | NSD 0.065295
2023-08-15 14:47:07,196 -  Case ./data/Task10_Colon/imagesTr/colon_026.nii.gz - Dice 0.015407 | NSD 0.065295
[14:47:12.848]  Case ./data/Task10_Colon/imagesTr/colon_030.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:12,848 -  Case ./data/Task10_Colon/imagesTr/colon_030.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:20.227]  Case ./data/Task10_Colon/imagesTr/colon_040.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:20,227 -  Case ./data/Task10_Colon/imagesTr/colon_040.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:30.019]  Case ./data/Task10_Colon/imagesTr/colon_046.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:30,019 -  Case ./data/Task10_Colon/imagesTr/colon_046.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:39.798]  Case ./data/Task10_Colon/imagesTr/colon_091.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:39,798 -  Case ./data/Task10_Colon/imagesTr/colon_091.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:46.366]  Case ./data/Task10_Colon/imagesTr/colon_095.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:46,366 -  Case ./data/Task10_Colon/imagesTr/colon_095.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:53.232]  Case ./data/Task10_Colon/imagesTr/colon_098.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:53,232 -  Case ./data/Task10_Colon/imagesTr/colon_098.nii.gz - Dice 0.000000 | NSD 0.000000
[14:47:59.353]  Case ./data/Task10_Colon/imagesTr/colon_104.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:47:59,353 -  Case ./data/Task10_Colon/imagesTr/colon_104.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:07.321]  Case ./data/Task10_Colon/imagesTr/colon_129.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:07,321 -  Case ./data/Task10_Colon/imagesTr/colon_129.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:15.462]  Case ./data/Task10_Colon/imagesTr/colon_133.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:15,462 -  Case ./data/Task10_Colon/imagesTr/colon_133.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:25.344]  Case ./data/Task10_Colon/imagesTr/colon_136.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:25,344 -  Case ./data/Task10_Colon/imagesTr/colon_136.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:32.039]  Case ./data/Task10_Colon/imagesTr/colon_141.nii.gz - Dice 0.000000 | NSD 0.017542
2023-08-15 14:48:32,039 -  Case ./data/Task10_Colon/imagesTr/colon_141.nii.gz - Dice 0.000000 | NSD 0.017542
[14:48:38.911]  Case ./data/Task10_Colon/imagesTr/colon_142.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:38,911 -  Case ./data/Task10_Colon/imagesTr/colon_142.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:44.344]  Case ./data/Task10_Colon/imagesTr/colon_145.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:44,344 -  Case ./data/Task10_Colon/imagesTr/colon_145.nii.gz - Dice 0.000000 | NSD 0.000000
[14:48:53.632]  Case ./data/Task10_Colon/imagesTr/colon_163.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:48:53,632 -  Case ./data/Task10_Colon/imagesTr/colon_163.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:00.167]  Case ./data/Task10_Colon/imagesTr/colon_164.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:00,167 -  Case ./data/Task10_Colon/imagesTr/colon_164.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:06.922]  Case ./data/Task10_Colon/imagesTr/colon_181.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:06,922 -  Case ./data/Task10_Colon/imagesTr/colon_181.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:14.982]  Case ./data/Task10_Colon/imagesTr/colon_194.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:14,982 -  Case ./data/Task10_Colon/imagesTr/colon_194.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:21.982]  Case ./data/Task10_Colon/imagesTr/colon_202.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:21,982 -  Case ./data/Task10_Colon/imagesTr/colon_202.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:26.959]  Case ./data/Task10_Colon/imagesTr/colon_205.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:26,959 -  Case ./data/Task10_Colon/imagesTr/colon_205.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:33.510]  Case ./data/Task10_Colon/imagesTr/colon_207.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:33,510 -  Case ./data/Task10_Colon/imagesTr/colon_207.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:41.245]  Case ./data/Task10_Colon/imagesTr/colon_208.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:41,245 -  Case ./data/Task10_Colon/imagesTr/colon_208.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:50.006]  Case ./data/Task10_Colon/imagesTr/colon_215.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:50,006 -  Case ./data/Task10_Colon/imagesTr/colon_215.nii.gz - Dice 0.000000 | NSD 0.000000
[14:49:57.969]  Case ./data/Task10_Colon/imagesTr/colon_218.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:57,969 -  Case ./data/Task10_Colon/imagesTr/colon_218.nii.gz - Dice 0.000000 | NSD 0.000000
2023-08-15 14:49:58,035 - - Test metrics Dice: 0.00059259165
2023-08-15 14:49:58,035 - - Test metrics NSD: 0.0031860285794686172

Sorry we just did some modification to the training and testing code, but the checkpoint is not updated yet. I will update the checkpoint within this week.

So, is the training and test code provided on GitHub the version before or after your update?

@peterant330
Copy link
Collaborator

ing and test

The code is after the updating. But the checkpoint is before the updating (where the architecture is actually one of the ablation). Sorry very much for the inconvenience.

@CunminZhao
Copy link

ing and test

The code is after the updating. But the checkpoint is before the updating (where the architecture is actually one of the ablation). Sorry very much for the inconvenience.

thanks for your reply

@bingcheg
Copy link

Dear author

Have you updated the latest checkpoint so that I can reproduce your results?

@peterant330
Copy link
Collaborator

Dear author

Have you updated the latest checkpoint so that I can reproduce your results?

Hi,

I just uploaded checkpoint for MSD-pancreas, lits and MSD-Colon. The training of Kits takes longer and will be uploaded soon.

Best

@amparobetancourt
Copy link

Dear author, thanks for your contribution!
I downloaded the latest lits checkpoint to reproduce your results on the test data, however I can't load the weights I keep getting an error
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

Which doesnt happen with the MSD-pancreas weights and I have the recommended versions and packages , do you have any advice on this ?
Thanks in advance!

@peterant330
Copy link
Collaborator

Dear author, thanks for your contribution! I downloaded the latest lits checkpoint to reproduce your results on the test data, however I can't load the weights I keep getting an error RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

Which doesnt happen with the MSD-pancreas weights and I have the recommended versions and packages , do you have any advice on this ? Thanks in advance!

Hi,
I think this is the error due to the broken file. I am unsure if your downloading process breaks the file or if I uploaded the broken file. I just uploaded it again and maybe you can have a try.

@amparobetancourt
Copy link

Thank you! It works now!

@ghost
Copy link

ghost commented Sep 20, 2023

Dear author, I also meet the similar problem when using the pretrained checkpoint for the MSD-colon, it says "RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory" Can you please make sure the uploaded checkpoint for the MSD-colon is correct?

Thank you so much

@peterant330
Copy link
Collaborator

Dear author, I also meet the similar problem when using the pretrained checkpoint for the MSD-colon, it says "RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory" Can you please make sure the uploaded checkpoint for the MSD-colon is correct?

Thank you so much

Hi, we have uploaded a new one. Can you try again?

@yun263214678
Copy link

yun263214678 commented Oct 14, 2023 •
edited
Loading

2023-10-14 19:53:25,603 - - Test metrics Dice: 0.58578235
2023-10-14 19:53:25,603 - - Test metrics NSD: 0.7426014035058957

Hi, thanks a lot for your work, I tested colon with --num_prompts 1 , but in the test.py, can we use rand_crop_size in test data? which means that a volume has more than one point, and should the testing phase be better at the original size? (no rand_crop) . Or comparative experiments with other models, for example, does nnU-Net also use random crop size when testing?

@peterant330
Copy link
Collaborator

2023-10-14 19:53:25,603 - - Test metrics Dice: 0.58578235 2023-10-14 19:53:25,603 - - Test metrics NSD: 0.7426014035058957

Hi, thanks a lot for your work, I tested colon with --num_prompts 1 , but in the test.py, can we use rand_crop_size in test data? which means that a volume has more than one point, and should the testing phase be better at the original size? (no rand_crop) . Or comparative experiments with other models, for example, does nnU-Net also use random crop size when testing?

Hi, for the test, we also use rand_crop_size, which is set to be the same as during the training. nnU-Net using a sliding window inference mechanism where the prediction is also predicted on a patch-wise basis but all the patches in the image are involved for inference. Our method requires a point as a prompt, as a result, we cannot predict for those patches far away from the prompt. Only patches with prompts inside can be used for prediction. Actually, this is also one advantage of interactive segmentation, we have knowledge about where the foreground is by prompts.

@yun263214678
Copy link

2023-10-14 19:53:25,603 - - Test metrics Dice: 0.58578235 2023-10-14 19:53:25,603 - - Test metrics NSD: 0.7426014035058957
Hi, thanks a lot for your work, I tested colon with --num_prompts 1 , but in the test.py, can we use rand_crop_size in test data? which means that a volume has more than one point, and should the testing phase be better at the original size? (no rand_crop) . Or comparative experiments with other models, for example, does nnU-Net also use random crop size when testing?

Hi, for the test, we also use rand_crop_size, which is set to be the same as during the training. nnU-Net using a sliding window inference mechanism where the prediction is also predicted on a patch-wise basis but all the patches in the image are involved for inference. Our method requires a point as a prompt, as a result, we cannot predict for those patches far away from the prompt. Only patches with prompts inside can be used for prediction. Actually, this is also one advantage of interactive segmentation, we have knowledge about where the foreground is by prompts.

Thanks for your reply, and I also want to know that if we need same crop size for different models in both train and test step, does the comparative experiment need to be set like this, because it seems that crop size has a great impact on the result during testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@peterant330 @bingcheg @amparobetancourt @CunminZhao @ryylcc @yun263214678