add 2024-08-16-morphological-data post

_posts/2024-08-16-morphological-data.md

@@ -0,0 +1,99 @@
+---
+layout: post
+title:  SourceExtractor++ Morphological Catalogs
+date:   2024-08-16 13:36:32 +0200
+categories: imaging
+tags: release catalog jwst
+author: Aurélien Genin
+showOnHighlights: true
+---
+{% include components/tags.html %}
+
+<!-- ![png]({{ site.baseurl }}/assets/post_files/2024-08-16-morphological-data/data-model-resid.png) -->
+
+<img src="{{ site.baseurl }}/assets/post_files/2024-08-16-morphological-data/data-model-resid.png" alt="data-model-resid" style="width:100%;"/>
+
+Here we summarize the files available for morphological data. This extension to the DJA provides valuable morphological data for over 340k galaxies in the following fields :
+* ceers-full-grizli-v7.2
+* gds-grizli-v7.2
+* gdn-grizli-v7.3
+* primer-uds-north-grizli-v7.2
+* primer-uds-south-grizli-v7.2
+* primer-cosmos-east-grizli-v7.0
+* primer-cosmos-west-grizli-v7.0
+
+The morphologies of these galaxies have been measured using both Sérsic and Bulge+Disk model. It made use of [SourceXtractor++](https://github.com/astrorama/SourceXtractorPlusPlus) to run the model fitting.
+
+A merged catalog of all of the processed fields is available at [full-good_morpho-phot.fits.gz](https://s3.amazonaws.com/aurelien-sepp/full-good_morpho-phot.fits.gz) (1.2 Gb).  The first extension provides the merged DJA photometry and stellar population properties estimated from the photometric redshift fit.  The second extension provides the morphology measurements of the Sersic model fit and the third extension provides parameters of the Bulge+Disk (B+D) morphology fit.
+
+# File extensions
+
+For a given `field`, the following files are available:
+
+* `{field}_morpho-phot.fits.gz`: **Merged catalogs of DJA photometry, EAZY SED fitting (from the DJA) and SE++ fittings in Sérsic and B+D models.** They are stored in 3 HDUs inside the FITS file (DJA phot+EAZY, Sérsic model, B+D model) where the rows of each table correspond to one another. A `flag` column was added in the merged DJA catalog to flag sources non-detected by SE++ (`0`), potential artifcats (`1`) and low SNR or magnitude above the limiting mag in each field (`3`). **Sources considered good for phyiscal study have a `flag` value of `2`.**
+
+* `(model|resid)_{fit}_{field}*.fits.gz`: **Model or residual images generated by SE++ with the `fit` model for different bands.** They have the exact same WCS as the images on the DJA.
+
+* `{field}*star_psf.psf`: **PSFs generated for each field in each filter band using PSFEx**. They made use of a custom star selection to give better results than the native auto-selection from PSFEx. This selection is based on the MU_MAX/MAG_AUTO plane, a method adapted from [Leauthaud et al. 2007, Section 3.6](https://ui.adsabs.harvard.edu/abs/2007ApJS..172..219L/abstract).
+
+You will also find the following file, non-relative to a specific field:
+
+* [`full-good_morpho-phot.fits.gz`](https://s3.amazonaws.com/aurelien-sepp/ceers-full-grizli-v7.2/catalog/ceers-full-grizli-v7.2_morpho-phot.fits.gz): **Joint catalog of all the `morpho-phot` catalogs in the different fields, keeping only the good sources (`flag` value of `2`)**. *Useful for plots without having to open every catalog individually*
+
+
+# File organization
+
+All the files for each field are available in links in the table below.
+
+| Field | Catalog | Sérsic images | B+D images | PSFs |
+| :---: | :---: | :---: | :---: | :---: |
+| ceers-full-grizli-v7.2 | [Catalog](https://s3.amazonaws.com/aurelien-sepp/ceers-full-grizli-v7.2/catalog/ceers-full-grizli-v7.2_morpho-phot.fits.gz) | [Sérsic](https://s3.amazonaws.com/aurelien-sepp/ceers-full-grizli-v7.2/sepp/sersic_rg4/checkimages/index.html) | [B+D](https://s3.amazonaws.com/aurelien-sepp/ceers-full-grizli-v7.2/sepp/B%2BD/checkimages/index.html) | [PSF](https://s3.amazonaws.com/aurelien-sepp/ceers-full-grizli-v7.2/psfex/index.html) |
+| gds-grizli-v7.2 | [Catalog](https://s3.amazonaws.com/aurelien-sepp/gds-grizli-v7.2/catalog/gds-grizli-v7.2_morpho-phot.fits.gz) | [Sérsic](https://s3.amazonaws.com/aurelien-sepp/gds-grizli-v7.2/sepp/sersic_rg4/checkimages/index.html) | [B+D](https://s3.amazonaws.com/aurelien-sepp/gds-grizli-v7.2/sepp/B%2BD/checkimages/index.html) | [PSF](https://s3.amazonaws.com/aurelien-sepp/gds-grizli-v7.2/psfex/index.html) |
+| gdn-grizli-v7.3 | [Catalog](https://s3.amazonaws.com/aurelien-sepp/gdn-grizli-v7.3/catalog/gdn-grizli-v7.3_morpho-phot.fits.gz) | [Sérsic](https://s3.amazonaws.com/aurelien-sepp/gdn-grizli-v7.3/sepp/sersic_rg4/checkimages/index.html) | [B+D](https://s3.amazonaws.com/aurelien-sepp/gdn-grizli-v7.3/sepp/B%2BD/checkimages/index.html) | [PSF](https://s3.amazonaws.com/aurelien-sepp/gdn-grizli-v7.3/psfex/index.html) |
+| primer-uds-north-grizli-v7.2 | [Catalog](https://s3.amazonaws.com/aurelien-sepp/primer-uds-north-grizli-v7.2/catalog/primer-uds-north-grizli-v7.2_morpho-phot.fits.gz) | [Sérsic](https://s3.amazonaws.com/aurelien-sepp/primer-uds-north-grizli-v7.2/sepp/sersic_rg4/checkimages/index.html) | [B+D](https://s3.amazonaws.com/aurelien-sepp/primer-uds-north-grizli-v7.2/sepp/B%2BD/checkimages/index.html) | [PSF](https://s3.amazonaws.com/aurelien-sepp/primer-uds-north-grizli-v7.2/psfex/index.html) |
+| primer-uds-south-grizli-v7.2 | [Catalog](https://s3.amazonaws.com/aurelien-sepp/primer-uds-south-grizli-v7.2/catalog/primer-uds-south-grizli-v7.2_morpho-phot.fits.gz) | [Sérsic](https://s3.amazonaws.com/aurelien-sepp/primer-uds-south-grizli-v7.2/sepp/sersic_rg4/checkimages/index.html) | [B+D](https://s3.amazonaws.com/aurelien-sepp/primer-uds-south-grizli-v7.2/sepp/B%2BD/checkimages/index.html) | [PSF](https://s3.amazonaws.com/aurelien-sepp/primer-uds-south-grizli-v7.2/psfex/index.html) |
+| primer-cosmos-west-grizli-v7.0 | [Catalog](https://s3.amazonaws.com/aurelien-sepp/primer-cosmos-west-grizli-v7.0/catalog/primer-cosmos-west-grizli-v7.0_morpho-phot.fits.gz) | [Sérsic](https://s3.amazonaws.com/aurelien-sepp/primer-cosmos-west-grizli-v7.0/sepp/sersic_rg4/checkimages/index.html) | [B+D](https://s3.amazonaws.com/aurelien-sepp/primer-cosmos-west-grizli-v7.0/sepp/B%2BD/checkimages/index.html) | [PSF](https://s3.amazonaws.com/aurelien-sepp/primer-cosmos-west-grizli-v7.0/psfex/index.html) |
+| primer-cosmos-east-grizli-v7.0 | [Catalog](https://s3.amazonaws.com/aurelien-sepp/primer-cosmos-east-grizli-v7.0/catalog/primer-cosmos-east-grizli-v7.0_morpho-phot.fits.gz) | [Sérsic](https://s3.amazonaws.com/aurelien-sepp/primer-cosmos-east-grizli-v7.0/sepp/sersic_rg4/checkimages/index.html) | [B+D](https://s3.amazonaws.com/aurelien-sepp/primer-cosmos-east-grizli-v7.0/sepp/B%2BD/checkimages/index.html) | [PSF](https://s3.amazonaws.com/aurelien-sepp/primer-cosmos-east-grizli-v7.0/psfex/index.html) |
+
+
+# Code and catalog creation
+
+All the code used to create these catalogs can be found in the [DJA-SEpp](https://github.com/AstroAure/DJA-SEpp) GitHub repository. It contains:
+
+* The code of the [dja_sepp](https://pypi.org/project/dja-sepp/) package, made and used to generated these catalogs.
+
+* Scripts to:
+    * run PSFEx on the images available on the DJA to generated empirical PSFs,
+    * tile the images for SourceXtractor++,
+    * run SourceXtractor++,
+    * perform these steps autonomously on AWS EC2 instances.
+
+* Notebooks to:
+    * run locally PSFEx and SourceXtractor++,
+    * merge the tiled catalogs and images generated by SourceXtractor++,
+    * merge the morphological catalogs generated by SourceXtractor++ with photometric catalogs from the DJA,
+    * create plots to analyze the resulting catalogs,
+    * look at specific sources.
+
+The [README](https://github.com/AstroAure/DJA-SEpp?tab=readme-ov-file#readme) file in the GitHub repository presents the workflow to run the steps used for this release. It can be used to perform this morphological study on different fields available on the DJA, or be adapated to different needs.
+
+
+# Completeness
+
+The following table presents the galaxies whose morphology has been measured in the different fields used for this release.
+
+| Field | DJA | Sérsic | Bulge+Disk | Both |
+| :---: | :---: | :---: | :---: | :---: |
+| **CEERS** | 67035 | 52604 (78.5%) | 59046 (88.1%) | 51329 (76.6%) |
+**GOODS-S** | 57355 | 44931 (78.3%) | 52754 (92.0%) | 44016 (76.7%) |
+**GOODS-N** | 65481 | 53291 (81.4%) | 58852 (89.9%) | 51465 (78.6%) |
+**PRIMER-UDS (N)** | 68857 | 58947 (85.6%) | 67134 (97.5%) | 57945 (84.2%) |
+**PRIMER-UDS (S)** | 65864 | 57397 (87.1%) | 64537 (98.0%) | 56476 (85.7%) |
+**PRIMER-COSMOS (E)** | 50655 | 42359 (83.6%) | 48496 (95.7%) | 41597 (82.1%) |
+**PRIMER-COSMOS (W)** | 51362 | 40493 (78.8%) | 46964 (91.4%) | 39704 (77.3%) |
+**Total** | **426609** | **350022 (82.0%)** | **397783 (93.2%)** | **342892 (80.4%)** |
+
+
+# Technical details
+
+For more information please check the [Internship report written by Aurélien Genin]({{ site.baseurl }}/assets/post_files/2024-08-16-morphological-data/Rapport_DAWN_AurelienGENIN2.pdf). You can find details on the PSF estimation (3.2), the models used (3.3.1), the workings of SourceXtractor++ (3.3.2), the implementation on AWS (3.4) and some premilinary results (4) .