The goal of this hack is to use Diffusion Models to build a simulations-driven prior that can be used to denoise and deconvolve galaxy images from JWST.
We would follow these steps:
-
Gather simulated galaxy images from the Illustris TNG hydrodynamical simulations to act as a prior on what perfect observations would look like:
-
Train a Score-Based Denoising Diffusion Model on these images. This is a type of generative models which can learn to generate images similar to the input training data. These diffusion models work by learning a mapping from pure noise to images in the training set:
-
Use the knownledge about galaxy morphology collected by the model to solve the inverse problem of reconstructing a high resolution, PSF-deconvolved, and noise-free image of observed JWST images similar to these:
In more details, we will be using a forward model of the observed images, which takes into account the noise and and PSF properties of the image, and solve a Bayesian inverse problem with the Diffusion Model as our prior, to sample posterior images, compatible with the observations and still likely under the prior.
You can watch the hack presentation by Marc during the telecon.
We will be using galaxy images from Illustris TNG, prepared with a radiative transfer code. The observations we will try to denoise and deconvolve are galaxy images from JWST.
- A nice introduction to diffusion models: https://lilianweng.github.io/posts/2021-07-11-diffusion-models/
- An example of paper using diffusion models as priors for inverse models:https://arxiv.org/abs/2201.05561