JWST Build 7.9.3

This patch release includes fixes for set_telescope_pointing.

JWST Build 7.9.1

The default COARSE tracking algorithm, COARSE_TR_202111, was updated to use FGS1 exclusively.

JWST Build 7.9 rc3

What's Changed

• Open changelog for development by @stscieisenhamer in #6555
• Update CRDS dependency by @stscieisenhamer in #6567
• SPB-778 Bugfix: JWST dependency uploads occuring with failed regressions by @shahm0 in #6568
• JP-2277: Add a blurb to step docs with external calls to stcal by @tapastro in #6566
• update version of stcal to 0.6.0 by @cshanahan1 in #6570
• JP-2384: Update names of MIRI coronagraphic dither patterns by @tapastro in #6573
• Prepare changelog for release by @nden in #6574
• open change log for post 1.4.1 development by @nden in #6575
• JP-2386: Added MIRI-MRS cross band options to outlier detection by @jemorrison in #6666
• JP-2024: Update default value of skymatch's 'skymethod' step parameter by @tapastro in #6580
• JP-2495: Change restriction on Level-2b association creation to nrc_image instead of nrc_tacq by @tapastro in #6681
• JP-2385: Changed logic in validation of grism bounding boxes by @tapastro in #6579
• Prepare for 1.4.2 by @stscieisenhamer in #6682

New Contributors

• @shahm0 made their first contribution in #6568

Full Changelog: 1.4.0...1.4.2

JWST Build 7.9 rc2

1.4.1 (2022-01-15)

dark_current

• Added docs mention of external algorithm in stcal [#6566]

datamodels

• Update names of MIRI coronagraphic flats dither patterns [#6573]

jump

• Added docs mention of external algorithm in stcal [#6566]

• Fix issue in jump detection that occurred when there were only 2 usable
  differences with no other groups flagged. This PR also added tests and
  fixed some of the logging statements in two-point difference. [spacetelescope/stcal#74]

linearity

• Added docs mention of external algorithm in stcal [#6566]

ramp_fitting

• Added docs mention of external algorithm in stcal [#6566]

saturation

• Added docs mention of external algorithm in stcal [#6566]

JWST Build 7.9 rc1

1.4.0 (2022-01-10)

ami_analyze

• Call img_median_replace to replace NaN's and pixels flagged with
  DO_NOT_USE in the input image. [#6334]

assign_wcs

• Open the specwcs reference file for WFSS modes using the with context
  manager. [#6160]

• Fix bug in NIRspec where bounding_box can be oversized in height for
  some of the slits. [#6257]

• Updated create_grism_bbox to be more robust against failures caused by
  bad input data. [#6309]

• Added a function that, when given RA, Dec, lambda, computes which ones project
  into a given NIRSpec IFU slice. [#6316]

• Changed in_ifu_slice in util.py to return the indices of elements in slice.
  Also the x tolerance on finding slice elements was increased. [#6326]

• Fix a bug due to which, under certain circumstances, PC-matrix and
  CDELT coefficients may be removed from FITS WCS of data products. [#6453]

• Fixed bug in NIRSpec MOS slitlet meta data calculations for background slits
  consisting of multiple shutters. [#6454]

• Enabled spectral order as input to WCS for NIRISS SOSS mode. [#6496]

associations

• Remove MIR_FLATMRS from Asn_Lv3MIRMRS rule. [#6548]

• Fix bug causing pytest to encounter an error in test collection when
  running with recent commits to astropy main (5.0.dev). [#6176]

• Enhanced level-2b ASN rules for NIRSpec internal lamp exposures to
  handle certain opmode/grating/lamp combinations that result in no data
  on one of the detectors. [#6304]

• Removed Constraint_ExtCal from Asn_Lv2WFSC constraints, as it was
  redundant with Constraint_Image_Science present. [#6384]

• Added constraint to Asn_Lv2ImageNonScience to prevent creation of asns
  for NRC_TACQ exposures with WFSC_LOS_JITTER in the DMS_NOTE. Also added
  new reduce method, Constraint.notall [#6404]

barshadow

• Modify computation of correction array to remove dependencies on the
  centering of the source within the slitlet, because for extended/uniform
  sources the centering is irrelevant. This fixes bugs encountered when
  computing the correction for background signal contained within slits with
  an off-center point source. [#6454, #6459]

cube_build

• Fix bug when creating cubes using output_type=channel. [#6138]

• Move computationally intensive routines to C extensions and
  removed miri psf weight function. [#6093]

• Moved cube blotting to a C extension [#6256]

• Moved variable definitions to top of code in C extension to
  support changes in #6093. [#6255]

• Added weighting option driz (3D drizzling) [#6297]

• Using assign_wsc.utils.in_ifu_slice function to determine which NIRSpec
  sky values mapped to each detector slice. [#6326]

• Fixed error in final exposure times calculated by blend headers. Only the input models
  used in the IFU cube are passed to blend headers. [#6360]

• Update of documentation to explain 3d drizzling and remove miri psf weighting [#6371]

• Fix a bug when creating internal_cal type cubes [#6398]

• Fix incorrect spatial footprint for single band MRS IFU cubes [#6478]

dark_current

• Refactored the code in preparation for moving the code to STCAL. [#6336]

• Moved dark current code from JWST to STCAL. [spacetelecope/stcal#63] [#6444]

• Updated step docs to explain unique form of MIRI dark reference data [#6529]

datamodels

• Remove astropy.io registration of JwstDataModel. [#6179]

• Update VELOSYS keyword comment [#6298]

• Added new keywords FPE_SIDE and ICE_SIDE to core schema [#6314]

• Fix bug preventing extra arguments when calling datamodels.open
  on an ASDF file. [#6327]

• Implement memmap argument when calling datamodels.open on an ASDF
  file. [#6327]

• Fix bug in schema that disallowed valid p_grating values. [#6333]

• Add NDArrayType to list of valid types for RegionsModel.regions. [#6333]

• Fix a bug in wcs_ref_models where SpecwcsModel was failing the SimpleModel
  validation as it contains a list of models rather than one simple model.
  Also add some missing allowed BAND values for MIRI MRS distortion
  and regions files. Fix an incorrect comment on
  FilteroffsetModel. [#6362]

• Changed reference file model name from ResidualFringeModel to
  FringeFreq [#6385]

• Updated data products documentation to indicate that variance and error arrays
  are now included in resampled products. [#6420]

• Added SOSS-specific extraction parameters to core schema; add new
  datamodel to store SOSS model traces and aperture weights [#6422]

• Added the MirLrsPathlossModel for use in the ``pathloss` step. [#6435]

• Added new column 'reference_order' to 'planned_star_table' in
  guider_raw and guider_cal schemas [#6368]

• Moved new column 'reference_order' in guider schemas' planned
  star table to second in order, after 'guide_star_order' [#6465]

• Updated moving_target schema changing mt_detector_x/y to mt_sci_x/y [#6485]

• Fixed names of NIRISS SOSS extract_1d parameter keywords to be legal FITS [#6499]

• Update PATTTYPE enum values to match spellings used in keyword dictionary [#6501]

• Updated documentation to point to stdatamodels.util
  for calls to create_history_entry [#6537]

• Added keyword EXP_TYPE to PsfMaskModel schema [#6540]

• Updated FILTEROFFSET reference file docs to add NIRCam information. [#6541]

dark_current

• Fixed bug during save of optional averaged darks output, bug with
  providing step a file instead of a datamodel, added regression test [#6450]

documentation

• Update text to point to the JWST CRDS website. [#6549]

extract_1d

• Updated to propagate SRCTYPE keyword during extraction of MIRI LRS
  fixed-slit inputs that are in SlitModel form. [#6212]

• Assign 0-indexed integration number to INT_NUM if input
  INT_TIMES table is empty. [#6369]

• Change INT_NUM assignment to 1-indexed. [#6388]

• Added NRS_LAMP as an exp_type that has the extract1d ref file in asdf format [#6460]

• Added the center_xy step argument to allow user-specified x/y
  center of IFU extraction apertures [#6503]

• Delivery of new algorithm ATOCA for SOSS extraction, along with four new reference
  files: speckernel, specprofile, spectrace and wavemap. [#6467]

• Added step parameter soss_atoca to turn ATOCA algorithm on, with box extraction
  the default algorithm [#6551]

flatfield

• Updated flatfield step docs to include complete details on how the
  variance and error arrays are updated. [#6245]

• Fixed a bug in flatfield for NIRSpec BrightObj mode where the S-flat cutout
  was calculated incorrectly by not accounting for the slit offset [#6332]

• Added check to NRS_LAMP exposures that routes imaging exposures to the imaging
  half of flatfield, where they will skip the step as expected [#6462]

jump

• Updated jump detection step to use common code moved to stcal [#6089]

• In stcal (pr #72), several changes were made to fix existing bugs in the
  twopoint difference routine for jump detection. Some of these issues
  resulted in jumps erroneously being flagged for pixels with only two
  usable groups (i.e one usable difference). This PR on the JWST side
  fixes one of the unit tests to account for this. [#6552]

lib

• Implement the MAST AUI interface to the Engineering Database. [#6288]

• Fix ROLL_REF and angle_to_vector calculations [#6452]

• Fix bad implementation of angle_to_vector in set_telescope_pointing. [#6452]

• Use TRACK algorithms for moving target exposures. [#6452]

• Move setting of the default method to calc_transforms. [#6482]

linearity

• Use the common code in STCAL for linearity correction. [#6386]

• Update of linearity test to support STCAL PR65 [#6509]

outlier_detection

• Revert back to using 'linear' interpolation method as default for blot.
  The bug in the implementation of the bilinear interpolator in the drizzle
  package is now fixed. [#6146]

• Log number of flagged outliers in outlier_detection [#6260]

pathloss

• Updated the pathloss step and documentation to include processing of
  MIRI LRS fixed-slit exposures. [#6435]

persistence

• Changed logger from root to __name__ [#6389]

pipeline

• Added wfss_contam step to calwebb_spec2 pipeline flow for WFSS modes [#6207]

• Changed logger from root to __name__ for Ami3, Detector1, Dark, and Guider
  Pipelines [#6389]

• Updated the calwebb_spec2 pipeline to apply the pathloss step to
  MIRI LRS fixed-slit exposures. [#6435]

ramp_fitting

• Fix special handling for 2 group ramp. [spacetelescope/stcal#70]

• Fix issue with inappropriately including a flagged group at the beginning
  of a ramp segment. [spacetelescope/stcal#68]

• Pixels with negative median rates will have VAR_POISSON set to zero.
  [spacetelescope/stcal#59]

• Update RampFitStep to pass DQ flags as a parameter to the ramp_fit
  algorithm code in stcal. Bump version requirement for stcal. [#6072]

refpix

• Refactored the subtract_reference routine for NRS IRS2 data to reduce
  memory usage. [#6356]

• Updated bad link to JDox in the step documentation. [#6372]

regtest

• Update okifying to handle full folder updates for associations [#6218]

• Remove default cfg usage from all relevant regtests; replaced with
  either pipeline alias or Step instance [#6391]

resample

• Refactor resample_spec to use a separate function for computing the output
  rectified WCS for lamp data. [#6296]

• Fix a crash in resample_spec due to undefined variance arrays. [#6305]

• Fix handling...

1.3.3

What's Changed

• Avoid photutils 1.2.0 [#6378]

JWST Build 7.8.2

Changes in jwst 1.3.2

associations

• Enhanced level-2b ASN rules for NIRSpec internal lamp exposures to
  handle certain opmode/grating/lamp combinations that result in no data
  on one of the detectors. [#6304]

cube_build

• Fix bug when creating cubes using output_type=channel. [#6138]

• Move computationally intensive routines to c extensions and
  removed miri psf weight function. [#6093]

• Moved variable definitions to top of code in c extension to
  support changes in #6093. [#6255]

• Moved cube blotting to a c extension [#6256]

pipeline

• Updated calwebb_tso3 to be more robust in handling null results from
  the tso_photometry step. [#6265]

JWST Build 7.8.1

Changes in jwst v1.3.1

lib

• Fixed a bug in set_telescope_pointing that was setting wrong meta for the pointing quality [#6264]

JWST Build 7.8.1

Changes in jwst v1.3.0

associations

• Ensure no Lv3_WFSC associations created on group candidates [#6131]

datamodels

• Add new PATTTYPE values for MIRI Coronagraphic flats:
  4QPM_LFLAT, 4QPM_PFLAT, LYOT_LFLAT, LYOT_PFLAT. [#6232]

• Update DarkModel to use uint32 for DQ array. [#6228]

• Add NOUTPUTS keyword to the DarkModel schema. [#6213]

lib

• Add overriding of the matrix calculations to set_telescope_pointing.py [#5843]

• Add guide star-based pointing algorithm to set_telescope_pointing.py [#5843]

resample

Fix the extreme memory consumption seen in resampling of variance arrays. [#6251]

tweakreg

• Add an upper tweak threshold of 10 arcsec to tweakreg [#6252]

wfs_combine

-- Add option to flip the dither locations so that images with different
filters will have the same pixel locations [#6101]

-- Fixed the refine option to correctly use the cross correlation to align
the images if the WCS is off [#6101]

JWST Build 7.8

Release Notes

The JWST calibration pipeline (v1.2.3) has been made available for installation on PyPi and github. A high-level list of updates in this build is given below.

Association generator rules updates

• Properly segregate NIRCam + NIRISS parallel Wide Field Slitless Spectroscopy (WFSS) exposures by instrument
• Process MIRI MRS dedicated background observations as stand-alone science
• Only associate direct images obtained using the same instrument channel with WFSS grism images
• Generate "spec3" associations for MIRI MRS observations that don't use a dither pattern
• Don't create ASN's for NIRCam Engineering Template exposures that use EXP_TYPE='NRC_GRISM' and PUPIL='CLEAR'
• Only use exposures from other primary nod positions as background members for NIRSpec fixed-slit observations that employ sub-pixel dithering

Updates to the Level-1b set_telescope_pointing script to:

• Moved the JWST_[XYZ] and JWST_D[XYZ] keywords from the primary header to the SCI extension header

Updates to Level-2a processing for all exposure modes include:

• Updated the "refpix" step to properly handle subarray exposures that use 4 amplifiers for readout
• Updated the "jump" step to detect jumps in integrations containing only 3 or 4 groups
• Refactored the Ordinary Least Squares (OLS) algorithms in the "ramp_fit" step

Updates to Level-2b processing for all exposure modes include:

• Fix the computation of S_REGION for exposures that straddle the RA=0 boundary
• Added new step parameter to the "background" step that allows saving the combined background image

Updates to Level-2b spectroscopic (calwebb_spec2) processing include:

• Updated the "srctype" step to populate SRCTYPE based on the "is_extended" flag for WFSS exposures
• "cube_build" step updates to improve processing efficiency
• Implemented variance and error propagation in the "extract_1d" step

Updates to the Level-3 imaging (calwebb_image3) processing include:

• Updated the "resample" step to propagate variance and error arrays
• Modified the "source_catalog" step to compute photometric errors
• Modified the "source_catalog" step to store "is_extended" instead of "is_star" for each source
• Modified the "source_catalog" step to compute concentration indexes as flux ratios instead of magnitude differences

Updates to the Level-3 spectroscopic (calwebb_spec3) processing include:

• Updated the "resample_spec" step to propagate variance and error arrays
• "cube_build" step updates to improve processing efficiency
• Implemented variance and error propagation in the "extract_1d" step

Updates to the Level-3 TSO (calwebb_tso3) pipeline include:

• Modified the "outlier_detection" step to use simple sigma clipping up the stack of integrations within an exposure

Updates to the Level-3 AMI (calwebb_ami3) pipeline include:

• Fix sign error in AMI pupil phase results

The complete change log follows:

datamodels

• Add back and use "CALCULATED" for ENGQLPTG. [#6135]

• Convert incoming Path objects to strings in datamodels.open [#6130]