add feature varnet (#340)

* add feature varnet

* Included copyright header and fixed linter tests

* Re-adding fastmri/models/feature_varnet.py to ensure CI passes

* Re-adding fastmri/models/feature_varnet.py to ensure CI passes - reinstalled black 22.3.0

* Re-adding fastmri/models/feature_varnet.py to ensure compatibility with mypy 1.1.1

* Change fastMRI version

* Fix black

* Move feature_varnet to fastmri_examples

* Reformat feature_varnet

* Move module imports

* Reorganize imports

---------

Co-authored-by: Matthew Muckley <[email protected]>

LIST_OF_PAPERS.md

@@ -17,6 +17,7 @@ The following is a short list of fastMRI publications. Clicking on the title wil
 13. Bakker, T., Muckley, M.J., Romero-Soriano, A., Drozdzal, M. & Pineda, L. (2022). [On learning adaptive acquisition policies for undersampled multi-coil MRI reconstruction](#on-learning-adaptive-acquisition-policies-for-undersampled-multi-coil-mri-reconstruction). In * *International Conference on Medical Imaging with Deep Learning*, pages 63-85.
 14. Radmanesh, A.\*, Muckley, M. J.\*, Murrell, T., Lindsey, E., Sriram, A., Knoll, F., ... & Lui, Y. W. (2022). [Exploring the Acceleration Limits of Deep Learning VarNet-based Two-dimensional Brain MRI](#exploring-the-acceleration-limits-of-deep-learning-varnet-based-two-dimensional-brain-mri). *Radiology: Artificial Intelligence*, 4(6), page e210313.
 15. Johnson, Patricia M., Lin, D. J., Zbontar, J., Zitnick, C. L., Sriram, A., Mucklye, M., ..., & Knoll, F. (2023). [Deep learning reconstruction enables prospectively accelerated clinical knee MRI](#deep-learning-reconstruction-enables-prospectively-accelerated-clinical-knee-mri) *Radiology*, page 220425.
+16. Giannakopoulos, I. I., Muckley, M. J., Kim, J., Breen, M., Johnson, P. M., Lui, Y. W., & Lattanzi, R. (2024). [Accelerated MRI reconstructions via variational network and feature domain learning](#accelerated-mri-reconstructions-via-variational-network-and-feature-domain-learning) *Scientific Reports*, 14(1), 10991.
 
 ## fastMRI: An open dataset and benchmarks for accelerated MRI
 
@@ -355,3 +356,24 @@ In a clinical setting, deep learning reconstruction enabled a nearly twofold red
     doi = {10.1148/radiol.220425},
 }
 ```
+
+## Accelerated MRI reconstructions via variational network and feature domain learning
+
+[Publication](https://doi.org/10.1038/s41598-024-59705-0) [Code](https://github.com/facebookresearch/fastMRI/tree/main/fastmri_examples/feature_varnet)
+
+**Abstract**
+
+We introduce three architecture modifications to enhance the performance of the end-to-end (E2E) variational network (VarNet) for undersampled MRI reconstructions. We first implemented the Feature VarNet, which propagates information throughout the cascades of the network in an N-channel feature-space instead of a 2-channel feature-space. Then, we add an attention layer that utilizes the spatial locations of Cartesian undersampling artifacts to further improve performance. Lastly, we combined the Feature and E2E VarNets into the Feature-Image (FI) VarNet, to facilitate cross-domain learning and boost accuracy. Reconstructions were evaluated on the fastMRI dataset using standard metrics and clinical scoring by three neuroradiologists. Feature and FI VarNets outperformed the E2E VarNet for 4, 5 and 8 Cartesian undersampling in all studied metrics. FI VarNet secured second place in the public fastMRI leaderboard for 4 Cartesian undersampling, outperforming all open-source models in the leaderboard. Radiologists rated FI VarNet brain reconstructions with higher quality and sharpness than the E2E VarNet reconstructions. FI VarNet excelled in preserving anatomical details, including blood vessels, whereas E2E VarNet discarded or blurred them in some cases. The proposed FI VarNet enhances the reconstruction quality of undersampled MRI and could enable clinically acceptable reconstructions at higher acceleration factors than currently possible.
+
+```BibTeX
+@article{giannakopoulos2024accelerated,
+  title={Accelerated MRI reconstructions via variational network and feature domain learning},
+  author={Giannakopoulos, Ilias I and Muckley, Matthew J and Kim, Jesi and Breen, Matthew and Johnson, Patricia M and Lui, Yvonne W and Lattanzi, Riccardo},
+  journal={Scientific Reports},
+  volume={14},
+  number={1},
+  pages={10991},
+  year={2024},
+  publisher={Nature Publishing Group UK London}
+}
+```

README.md

@@ -122,6 +122,7 @@ in another repository.
   * [End-to-End Variational Networks for Accelerated MRI Reconstruction ({A. Sriram*, J. Zbontar*} et al., 2020)](https://github.com/facebookresearch/fastMRI/tree/master/fastmri_examples/varnet/)
   * [MRI Banding Removal via Adversarial Training (A. Defazio, et al., 2020)](https://github.com/facebookresearch/fastMRI/tree/master/banding_removal)
   * [Deep Learning Reconstruction Enables Prospectively Accelerated Clinical Knee MRI (P. Johnson et al., 2023)](https://github.com/facebookresearch/fastMRI/tree/main/fastmri_examples/RadiologyJohnson2022)
+  * [Accelerated MRI reconstructions via variational network and feature domain learning (I. Giannakopoulos et al., 2024)](https://github.com/facebookresearch/fastMRI/tree/main/fastmri_examples/feature_varnet)
 
 * **Active Acquisition**
   * (external repository) [Reducing uncertainty in undersampled MRI reconstruction with active acquisition (Z. Zhang et al., 2019)](https://github.com/facebookresearch/active-mri-acquisition/tree/master/activemri/experimental/cvpr19_models)
@@ -212,3 +213,5 @@ corresponding abstracts, as well as links to preprints and code can be found
 14. Radmanesh, A.\*, Muckley, M. J.\*, Murrell, T., Lindsey, E., Sriram, A., Knoll, F., ... & Lui, Y. W. (2022). [Exploring the Acceleration Limits of Deep Learning VarNet-based Two-dimensional Brain MRI](https://doi.org/10.1148/ryai.210313). *Radiology: Artificial Intelligence*, 4(6), page e210313.
 15. Johnson, P.M., Lin, D.J., Zbontar, J., Zitnick, C.L., Sriram, A., Muckley, M., Babb, J.S., Kline, M., Ciavarra, G., Alaia, E., ..., & Knoll, F. (2023). [Deep Learning Reconstruction Enables Prospectively Accelerated Clinical Knee MRI](https://doi.org/10.1148/radiol.220425). *Radiology*, 307(2), page e220425.
 16. Tibrewala, R., Dutt, T., Tong, A., Ginocchio, L., Keerthivasan, M.B., Baete, S.H., Lui, Y.W., Sodickson, D.K., Chandarana, H., Johnson, P.M. (2023). [FastMRI Prostate: A Publicly Available, Biparametric MRI Dataset to Advance Machine Learning for Prostate Cancer Imaging](https://arxiv.org/abs/2304.09254). *arXiv preprint, arXiv:2034.09254*.
+16. Giannakopoulos, I. I., Muckley, M. J., Kim, J., Breen, M., Johnson, P. M., Lui, Y. W., Lattanzi, R. (2024). [Accelerated MRI reconstructions via variational network and feature domain learning. Scientific Reports](https://www.nature.com/articles/s41598-024-59705-0). *Scientific Reports, 14(1), 10991*.
+

fastmri_examples/README.md

@@ -15,3 +15,4 @@ further details.
 
   * [End-to-End Variational Networks for Accelerated MRI Reconstruction ({A. Sriram*, J. Zbontar*} et al., 2020)](varnet/)
   * [On learning adaptive acquisition policies for undersampled multi-coil MRI reconstruction (T. Bakker et al., 2021)](adaptive_varnet/)
+  * [Accelerated MRI reconstructions via variational network and feature domain learning (I. Giannakopoulos et al., 2024)](feature_varnet/)

fastmri_examples/feature_varnet/README.md

@@ -0,0 +1,72 @@
+# Accelerated MRI reconstructions via variational network and feature domain learning
+
+This directory contains a PyTorch implementation for reproducing the following paper, to be published at MIDL 2022.
+
+[Accelerated MRI reconstructions via variational network and feature domain learning (I. Giannakopoulos, et al., 2024).][feature_varnet]
+
+## Installation
+We **strongly** recommend creating a separate conda environment for this example, as the
+PyTorch Lightning versions required differs from that of the base `fastmri` installation.
+
+Before installing dependencies, first install PyTorch according to the directions at the
+PyTorch Website for your operating system and CUDA setup
+(we used `torch` version 1.7.0 for our experiments). Then run
+
+```bash
+pip install -r fastmri_examples/feature_varnet/requirements.txt
+```
+
+
+## Example training commands:
+
+This code provides a few ablations of the end-to-end variational network, namely, feature varnet with weight sharing, feature varnet without weight sharing, attention feature varnet with weight sharing, feature-image varnet, and image-feature varnet. Train and test each model with the same commands as the end-to-end variational network and include an additional input argument to your input file:
+For the end-to-end varnet
+> --varnet_type e2e_varnet
+
+For the feature varnet with weight sharing
+> --varnet_type feature_varnet_sh_w
+
+For the feature varnet without weight sharing
+> --varnet_type feature_varnet_n_sh_w
+
+For the attention feature varnet with weight sharing
+> --varnet_type attention_feature_varnet_sh_w
+
+For the feature-image varnet
+> --varnet_type fi_varnet
+
+For the image-feature varnet
+> --varnet_type if_varnet
+
+See `train_feature_varnet.py` for additional arguments.
+
+
+## Example evaluation commands:
+
+Evaluate the model as the end-to-end varnet
+
+
+## Paths:
+
+Data and log paths are defined the fastmri_dirs.yaml
+
+
+## Citing
+
+If you use this this code in your research, please cite the corresponding
+paper:
+
+```BibTeX
+@article{giannakopoulos2024accelerated,
+  title={Accelerated MRI reconstructions via variational network and feature domain learning},
+  author={Giannakopoulos, Ilias I and Muckley, Matthew J and Kim, Jesi and Breen, Matthew and Johnson, Patricia M and Lui, Yvonne W and Lattanzi, Riccardo},
+  journal={Scientific Reports},
+  volume={14},
+  number={1},
+  pages={10991},
+  year={2024},
+  publisher={Nature Publishing Group UK London}
+}
+```
+
+[feature_varnet]: https://www.nature.com/articles/s41598-024-59705-0