Awesome Synthetic Medical Image

Welcome to the Synthetic Medical Imaging Repository – a comprehensive resource hub dedicated to the fascinating and rapidly evolving field of synthetic medical imaging. This repository is a side project of my master's thesis and aims to serve as a one-stop destination for researchers, developers, and enthusiasts in the field.

About This Repository

In this repository, you will find an extensive collection of significant literature and valuable GitHub repositories focused on synthetic medical imaging. Whether you are a seasoned researcher or a newcomer to the field, this resource is designed to provide you with a thorough understanding of current trends, methodologies, and breakthroughs in synthetic medical imaging.

Each category in this repository is meticulously curated, guiding you through the extensive field of synthetic medical imaging, from foundational concepts to the nuances of advanced modeling techniques.

Foundational Concepts and Theoretical Background

1. Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision Bobby Azad, Reza Azad, Sania Eskandari, Afshin Bozorgpour, Amirhossein Kazerouni, Islem Rekik, Dorit Merhof (2023)
2. A Comprehensive Survey of Deep Learning Research on Medical Image Analysis with Focus on Transfer Learning. SEMA ATASEVER, NUH AZGINOGLU, DUYGU SINANC TERZI, RAMAZAN TERZI (2022).
3. Overcoming barriers to data sharing with medical image generation: a comprehensive evaluation August DuMont Schütte, Jürgen Hetzel, Sergios Gatidis, Tobias Hepp, Benedikt Dietz, Stefan Bauer & Patrick Schwab. npj Digital Medicine volume 4, Article number: 141 (2021).
4. Federated learning and differential privacy for medical image analysis Mohammed Adnan, Shivam Kalra, Jesse C. Cresswell, Graham W. Taylor & Hamid R. Tizhoosh. Scientific Reports volume 12, Article number: 1953 (2022).
5. Multimodal Image Synthesis and Editing: The Generative AI Era Fangneng Zhan, Yingchen Yu, Rongliang Wu, Jiahui Zhang, Shijian Lu, Lingjie Liu, Adam Kortylewski, Christian Theobalt, Eric Xing
6. A Survey on Deep Generative 3D-aware Image Synthesis Weihao Xia, Jing-Hao Xue ACM Computing Surveys, Volume 56, Issue 4, November 2023, Article No.: 90, pp 1–34
7. RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model Fengxiang Bie, Yibo Yang, Zhongzhu Zhou, Adam Ghanem, Minjia Zhang, Zhewei Yao, Xiaoxia Wu, Connor Holmes, Pareesa Golnari, David A. Clifton, Yuxiong He, Dacheng Tao, Shuaiwen Leon Song Submitted on 2 Sep 2023
8. Image Generation: A Review Mohamed Elasri, Omar Elharrouss, Somaya Al-Maadeed & Hamid Tairi Volume 54, pages 4609–4646, (2022)
9. Generative AI for medical imaging analysis and applications Tanmai Sree Musalamadugu and Hemachandran Kannan, Preprint, December 2023.

Models

GANs

1. Generative Adversarial Networks Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio. Submitted on 10 Jun 2014.
2. High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, Bryan Catanzaro. Submitted on 30 Nov 2017 (v1), last revised 20 Aug 2018 (this version, v2).
3. Hierarchical Amortized GAN for 3D High Resolution Medical Image Synthesis Li Sun, Junxiang Chen, Yanwu Xu, Mingming Gong, Ke Yu, Kayhan Batmanghelich IEEE Journal of Biomedical and Health Informatics, Volume: 26, Issue: 8 Publisher: IEEE [GitHub]
4. GANs for Medical Image Synthesis: An Empirical Study. Skandarani, Y., Jodoin, P.-M., & Lalande, A. (2023). J. Imaging, 9(3), 69. University of Bourgogne Franche-Comte, University of Sherbrooke, University Hospital of Dijon.
5. Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks. Shin, H.-C., Tenenholtz, N. A., Rogers, J. K., Schwarz, C. G., Senjem, M. L., Gunter, J. L., Andriole, K., & Michalski, M. (2018).
6. StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesis Minguk Kang, Joonghyuk Shin, Jaesik Park (2022)
7. 3D cGAN based cross-modality MR image synthesis for brain tumor segmentation Biting Yu, Luping Zhou, Lei Wang, Jurgen Fripp, Pierrick Bourgeat , IEEE

Diffusion Models

1. Deep Unsupervised Learning using Nonequilibrium Thermodynamics Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, Surya Ganguli (2015)
2. Denoising Diffusion Probabilistic Models Jonathan Ho, Ajay Jain, Pieter Abbeel. Submitted on 19 Jun 2020 (v1), last revised 16 Dec 2020 (this version, v2).
3. Brain Imaging Generation with Latent Diffusion Models Walter H. L. Pinaya, Petru-Daniel Tudosiu, Jessica Dafflon, Pedro F da Costa, Virginia Fernandez, Parashkev Nachev, Sebastien Ourselin, M. Jorge Cardoso. Submitted on 15 Sep 2022.
4. Diffusion Models in Medical Imaging: A Comprehensive Survey. Kazerouni, A., Aghdam, E. K., Heidari, M., Azad, R., Fayyaz, M., Hacihaliloglu, I., & Merhof, D. (2023).
5. Medical Diffusion: Denoising Diffusion Probabilistic Models for 3D Medical Image Generation Firas Khader, Gustav Mueller-Franzes, Soroosh Tayebi Arasteh, Tianyu Han, Christoph Haarburger, Maximilian Schulze-Hagen, Philipp Schad, Sandy Engelhardt, Bettina Baessler, Sebastian Foersch, Johannes Stegmaier, Christiane Kuhl, Sven Nebelung, Jakob Nikolas Kather, Daniel Truhn (2022). Github
6. Investigating Data Memorization in 3D Latent Diffusion Models for Medical Image Synthesis. Salman Ul Hassan Dar, Arman Ghanaat, Jannik Kahmann, Isabelle Ayx, Theano Papavassiliu, Stefan O. Schoenberg, Sandy Engelhardt (2023).
7. Conversion of the Mayo LDCT Data to Synthetic Equivalent through the Diffusion Model for Training Denoising Networks with a Theoretically Perfect Privacy. Yongyi Shi, Ge Wang (2023).
8. EMIT-Diff: Enhancing Medical Image Segmentation via Text-Guided Diffusion Model. Zheyuan Zhang, Lanhong Yao, Bin Wang, Debesh Jha, Elif Keles, Alpay Medetalibeyoglu, Ulas Bagci (2023).
9. Synthetic Augmentation with Large-scale Unconditional Pre-training Jiarong Ye, Haomiao Ni, Peng Jin, Sharon X. Huang, Yuan Xue. Submitted on 8 Aug 2023.
10. A Survey of Diffusion Based Image Generation Models: Issues and Their Solutions Tianyi Zhang, Zheng Wang, Jing Huang, Mohiuddin Muhammad Tasnim, Wei Shi Submitted on 25 Aug 2023

Autorregresion (Transformers)

1. Autoregressive Image Generation using Residual Quantization Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, Wook-Shin Han. Submitted on 3 Mar 2022 (v1), last revised 9 Mar 2022 (this version, v2).
2. Morphology-preserving Autoregressive 3D Generative Modelling of the Brain by Petru-Daniel Tudosiu, Walter Hugo Lopez Pinaya, Mark S. Graham, Pedro Borges, Virginia Fernandez, Dai Yang, Jeremy Appleyard, Guido Novati, Disha Mehra, Mike Vella, Parashkev Nachev, Sebastien Ourselin, Jorge Cardoso. Submitted on 7 Sep 2022. Code
3. Scaling Autoregressive Models for Content-Rich Text-to-Image Generation by Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, Ben Hutchinson, Wei Han, Zarana Parekh, Xin Li, Han Zhang, Jason Baldridge, Yonghui Wu. Published: 07 Nov 2022
4. ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis by Patrick Esser, Robin Rombach, Andreas Blattmann, Bjorn Ommer. Part of Advances in Neural Information Processing Systems 34 (NeurIPS 2021).

Comparisons

1. Diffusion Probabilistic Models beat GANs on Medical Images Gustav Müller-Franzes, Jan Moritz Niehues, Firas Khader, Soroosh Tayebi Arasteh, Christoph Haarburger, Christiane Kuhl, Tianci Wang, Tianyu Han, Sven Nebelung, Jakob Nikolas Kather, Daniel Truhn [14th Dec., 2022] [arXiv, 2022]
2. Beware of Diffusion Models for Synthesizing Medical Images -- A Comparison with GANs in Terms of Memorizing Brain MRI and Chest X-Ray Images. Muhammad Usman Akbar, Wuhao Wang, Anders Eklund (2023).

Evaluation metrics

1. [Frechet Inception Distance (FID)]: GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Sepp Hochreiter. Submitted on 26 Jun 2017 (v1), last revised 12 Jan 2018 (this version, v6).
2. [Maximum Mean Discrepancy (MMD)]: A kernel two-sample test Gretton, A., et al. (2012). The Journal of Machine Learning Research, 13(1), pp.723-773.
3. [Structural Similarity Index Measure (SSIM)]: Image quality assessment: from error visibility to structural similarity Wang, Zhou, et al. (2004). IEEE Transactions on Image Processing, 13(4), pp.600-612.
4. [Learned Perceptual Image Patch Similarity (LPIPS)] : The Unreasonable Effectiveness of Deep Features as a Perceptual Metric Richard Zhang, Phillip Isola, Alexei A. Efros (UC Berkeley), Eli Shechtman, Oliver Wang (Adobe Research).
5. Peak Signal-to-Noise Ratio Wikipedia, Wikimedia Foundation, 22 Aug. 2022.\
6. A Review of the Image Quality Metrics used in Image Generative Models By Tabitha Oanda, a year ago.
7. Effectively Unbiased FID and Inception Score and where to find them by Min Jin Chong, David Forsyth. Submitted on 16 Nov 2019 (v1), last revised 15 Jun 2020 (this version, v3).

Conditional generation

1. Conditional Diffusion Models for Semantic 3D Medical Image Synthesis by Zolnamar Dorjsembe, Hsing-Kuo Pao, Sodtavilan Odonchimed, Furen Xiao. Submitted on 29 May 2023, last revised 31 Jul 2023.
2. Med-cDiff: Conditional Medical Image Generation with Diffusion Models by Alex Ling Yu Hung, Kai Zhao, Haoxin Zheng, Ran Yan, Steven S. Raman, Demetri Terzopoulos, and Kyunghyun Sung. Bioengineering 2023, 10(11), 1258.
3. Classifier-Free Diffusion Guidance by Jonathan Ho, Tim Salimans. Submitted on 26 Jul 2022.

Open-Source Medical imaging datasets

1. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions Philipp Tschandl, Cliff Rosendahl, Harald Kittler.

Implementation Guides and Tutorials

Contributing

We encourage contributions from all who are interested in synthetic medical imaging. If you have suggestions for additional resources, including literature, repositories, or tools, please submit a pull request or open an issue for discussion.

---

Thank you for visiting this repository. Let's drive forward the future of synthetic medical imaging together!