v0.18.1 – 2023-03-17

Summary

This is a bugfix release fixing three bugs and including some maintenance changes to the repository.

See the detailed changelog in the documentation and the list of merged PRs down below.

This release is also the first to include CEP 1, the definition of the CEP process, in the released ctapipe documentation.

Contributors

@LukasBeiske, @StFroese, @Tobychev, @kosack, @maxnoe and @nbiederbeck

What has changed since v0.18.0

Pull-requests containing changes of multiple nature are repeated.

• Render changelog for 0.18.1 (#2290) @maxnoe
• Fix wrong activity meta being written into output files (#2261) @maxnoe
• Only remove sort index from table if input did not contain it (#2288) @maxnoe
• Fix overwrite not working for ctapipe-apply-models (#2287) @maxnoe
• require lint CI before testing (#2281) @nbiederbeck
• Use pre-commit/action (#2276) @nbiederbeck
• Added favicon (#2263) @Tobychev
• Update CI config after default branch renaming (#2254) @maxnoe
• Add pypi badge to readme (#2253) @maxnoe

New features

• CEP-001: Definition of the proposal process (#2256) @Tobychev

Maintenance

• Add missing enter_context for TableLoader in train tools (#2286) @maxnoe
• CEP-001: Definition of the proposal process (#2256) @Tobychev
• Update pre-commit to isort 5.12.0 (#2271) @LukasBeiske
• Make devs aware of debugging and ipdb in docs (#2220) @StFroese
• Fix deprecation warning due to pkg_resources (#2259) @maxnoe

v0.18.0 – 2023-02-09

Summary

For the detailed changelog, please visit https://ctapipe.readthedocs.io/en/latest/changelog.html#ctapipe-v0-18-0-2023-02-09

Below, you can find a list of contributors and grouped list of all merged pull requests.

Contributors

@LukasBeiske, @LukasNickel, @RuneDominik, @StFroese, @Tobychev, @henningpt, @kosack, @maxnoe and @nbiederbeck

What has changed since v0.17.0

Pull-requests containing changes of multiple nature are repeated.

• Render 0.18 changelog using towncrier (#2251) @maxnoe
• Implement part of common reconstructor API (#2229) @maxnoe
• Fix wrong overwrite config in train_* tools (#2243) @maxnoe
• Simplify ChunkIterator, return start, stop, table (#2241) @maxnoe
• Notify users about test_plugin install (#2245) @RuneDominik
• Add HDF5Merger Component, use in MergeTool (#2179) @maxnoe
• Added a bit of clarification of how important classes attribute is to the tool doc (#2231) @Tobychev
• Implement transformation from GroundFrame to EarthLocation (#2167) @maxnoe
• Fix numba version to 0.56 (#2222) @maxnoe
• Move reading of stereo data before skipping empty events in HDF5EventSource (#2189) @maxnoe
• Rename towncrier bug to bugfix (#2202) @nbiederbeck
• Add coordinate overlay plotting to CameraDisplay (#2203) @maxnoe
• Fix unit handling in muon ring_completeness (#2197) @maxnoe
• Add changelogs for PRs merged since 0.17.0 (#2186) @maxnoe
• Fix version requirement for eventio, fixes #2185 (#2187) @maxnoe
• Fix byteorder fitshistogram (#2183) @maxnoe
• Add quality query to display dl1 tool (#2172) @maxnoe
• Run changelog check only on PRs, add support for no-changelog-needed … (#2176) @maxnoe
• Change default behaviour of run_tool (#2175) @nbiederbeck
• Fix docs build for traitlets 5.8 and array field defaults (#2173) @maxnoe
• Improve output of Tool in case of TraitError (#2171) @maxnoe
• Fix loading of observation info in TableLoader (#2163) @LukasBeiske
• Replace usage of deprecated astropy matrix functions (#2166) @maxnoe
• Make number of training events a telescope parameter (#2140) @maxnoe
• Don't use $HOME, use Path.home() instead (#2155) @kosack
• Error deprecationwarnings 2 (#2153) @nbiederbeck
• Improve missing field warning (#2148) @maxnoe
• Add ipython to docs requirements (#2150) @maxnoe
• Chunked model application (#2133) @maxnoe
• Allow lookup of telescope parameters by type (#2120) @maxnoe
• Fix flake8 complaint (#2137) @maxnoe
• fix set_line_hillas (#2134) @StFroese
• remove sort index after sorting (#2132) @maxnoe
• Allow using predicted energy in particle classifier (#2121) @maxnoe
• Speed up loader by using hstack instead of merge where possible (#2126) @maxnoe
• Speed up rotations, closes #2097 (#2098) @StFroese
• Update mailmap (#2117) @maxnoe
• Simplify apply_models (#2113) @maxnoe

New features

• Refactor apply tool to use HDF5Merger, read data only once (#2249) @maxnoe
• Disp reconstruction (#2138) @LukasBeiske
• Add option to configure ml prefixes (#2217) @LukasBeiske
• Improve likelihood muon intensity fitter (#2161) @StFroese
• Implement stereo trigger for subarray triggers (#2136) @maxnoe
• Support different frames for set_line_hillas (#2131) @StFroese
• Machine learning reconstruction (#1767) @maxnoe
• Add and use ComponentName traitlet instead of create_class_enum_trait, fixes #2100, #1874 (#2102) @maxnoe
• Allow overriding url via env variable (#2109) @maxnoe
• Atmosphere models for h_max to X_max conversion (#2000) @kosack

API Changes

• Raise error in case expected telescope-data is missing in HDF5EventSource (#2244) @RuneDominik
• Make muon analysis part of the processing tool (#2168) @StFroese

Bug Fixes

• Add rng attribute back to TrainEnergyRegressor (#2247) @maxnoe
• fix import error in metadata: fails on systems without pwd (#2152) @kosack
• Correctly setup components in dump-instrument-info, add test (#2106) @maxnoe

Refactoring and Optimization

• Use entry_points for plugins (#2101) @maxnoe
• Consistent overwrite (#2213) @LukasNickel
• Tool exit stack (#1926) @maxnoe
• Load observation info with TableLoader; standardize id fields in containers (#2096) @LukasBeiske
• Move impact_distance to coordinates to avoid circular import (#2095) @maxnoe

Maintenance

• Use entry_points for plugins (#2101) @maxnoe
• Consistent overwrite (#2213) @LukasNickel
• Use https links where possible (#2232) @maxnoe
• Fix sonarqube issues (#2215) @maxnoe
• Fix docstring in ImageProcessor (#2180) @maxnoe
• Update README (#2208) @StFroese
• Use weakref.proxy where parent=self. (#2223) @nbiederbeck
• Move test to correct module (#2225) @maxnoe
• Change actions for mamba / pip install methods. (#2207) @nbiederbeck
• Refactor initialization of telescope parameters, move to own module (#2190) @maxnoe
• Upload build docs as artifacts in CI (#2195) @nbiederbeck
• Fix frame name in coordinates warning (#2198) @maxnoe
• Create proper links to the PR in changelog (#2192) @maxnoe
• Raise error if run_tool encounters non-zero exit code and raises is True (#2191) @maxnoe
• Use towncrier for change log (#2144) @StFroese
• Add and use ComponentName traitlet instead of create_class_enum_trait, fixes #2100, #1874 (#2102) @maxnoe
• Update test data url (#2108) @maxnoe
• Only use setup.cfg, automatically include correct python requires in docs (#2093) @maxnoe
• Remove debug print (#2105) @maxnoe
• Fix warnings from deploy workflow (#2085) @maxnoe
• Update citations in README.rst (#2086) @maxnoe

v0.17.0 – 2022-09-27

Summary

There are three major changes in this release:

1. It is now possible to configure multiple reconstructors to be applied (at the moment, this means you can both run HillasReconstructor and HillasIntersection using ctapipe-process).

2. The naming scheme of telescope wise quantities at dl2 changed. In 0.16, e.g. the reconstructed impact distance to a specific telescope was HillasReconstructor_tel_distance, missing the prefix of the container the field was defined in. This is now fixed and the same column is now HillasReconstructor_tel_impact_distance.

3. The HillasReconstructor was optimized to remove pure-python loops over telescopes which resulted in a ca. 5x speed up when running on a La Palma prod6 run.

For smaller changes and bug fixes, see blow list of merged PRs.

Contributors

@Hckjs, @HealthyPear, @LukasNickel, @StFroese, @kosack, @maxnoe and @nbiederbeck

What has changed since v0.16.0

Pull-requests containing changes of multiple nature are repeated.

New features

• add --datamodel option to ctapipe-info (#2047) @kosack
• Merge DL2 tables. (#2050) @nbiederbeck

API Changes

• Improve ShowerProcessor API, fixes #2071 (#2073) @StFroese

Data Model Changes

• Use _impact_ instead of _tel_, fixes #2051 (#2060) @nbiederbeck

Bug Fixes

• Fix invalid case in HillasReconstructor and HillasIntersection (#2082) @maxnoe
• Convert float32 pi/2 value to float64 pi/2 value in SimTelEventSource (#2077) @maxnoe
• Allow data model version 4 in HDFEventSource, actually check which frame is used for hillas parameters (#2079) @maxnoe
• Fix unreachable assertion in Provenance unit test (#2072) @HealthyPear
• Fix wrong container used for camera frame in ImageProcessor, fixes #2062 (#2070) @maxnoe
• Fix tables.Node.__del__ throwing exceptions on interpreter exit when writing index tables, fixes #2067 (#2069) @maxnoe
• Sort unique cameras/types/optics to generate stable indices in hdf5, fixes #2063 (#2066) @maxnoe
• Make ExpressionEngine be no Component. Fixes #2065. (#2068) @nbiederbeck
• Fix camera rotation in examples/camera_rotation.py (#2056) @Hckjs
• Report HDF5EventSource as compatible for dl2 only files, fixes #2058 (#2059) @LukasNickel
• Merge simulated impact (#2049) @nbiederbeck
• Fix Container.as_dict for case of flatten=True and add_prefix=True (#1887) @maxnoe

Refactoring and Optimization

• Vectorize operations in HillasReconstructor (#2036) @maxnoe

Maintenance

• Change recommendation on reusing ArrayEvent in EventSources (#2076) @maxnoe
• Update mailmap (#2053) @maxnoe

v0.16.0 – 2022-08-01

Summary

This next major release of ctapipe includes a large number of improvements and new features, as well as significant changes to the API and data model that fix outstanding problems with both. Due to the nature of these required changes and to keep the code reasonably maintainable (in this pre-1.0 release), files written by earlier versions of ctapipe are not guaranteed to be readable with this one.

Contributors

@aknierim, @kosack, @maxnoe, @nbiederbeck, @satoshifukami0115 and @vuillaut

What has changed since v0.15.0

Pull-requests containing changes of multiple nature are repeated.

New features

• ObsConfig data model (#1973) @kosack
• Add abstract InvalidPixelHandler and NeighborAverage implementation, apply in CameraCalibrator and TwoPassWindowSum. Pass invalid pixels to extractors. (#2023) @maxnoe
• Get reference_location and optics/camera name from simtel metadata if available (#2018) @maxnoe
• Add methods to read run config and shower distribution to TableLoader (#2010) @maxnoe
• added a better template for release-drafter (#1880) @kosack
• Add another cleaning method in cleaning.py (#1672) @satoshifukami0115
• Round floats for hashing in instrument module, use __slots__ (#1986) @maxnoe
• Add location of subarray reference point to SubarrayDescription (#1980) @maxnoe
• recursively validate containers (#1976) @kosack
• Improve units code in hillas_parameters (#1988) @maxnoe
• Add TableLoader.read_telescope_events_by_type_chunked (#1984) @maxnoe
• Maintain event order of input file in TableLoader, fixes #1966 (#1968) @maxnoe
• Chunked table loader (#1935) @maxnoe
• Read dl2 in hdf5eventsource (#1940) @maxnoe
• add joined option to SubarrayDescription.to_table (#1939) @maxnoe
• compute impact distance to shower axis (#1899) @kosack
• Improved configuration of reference metadata (#1889) @kosack
• Allow passing already opened h5file to table loader (#1933) @maxnoe
• Add overwrite option to write_table (#1932) @maxnoe
• Support reading files created with extract_calibevent (#1929) @maxnoe
• improve array display (#1911) @kosack

API Changes

• Use n_ prefix consistently, (rename all num_ to n_), fixes #986 (#2046) @maxnoe
• Remove cutflow (#1766) @maxnoe
• Add pixel, channel, sample information to CameraReadout, update OpticsDescription (#2019) @maxnoe
• Remove support for by_type split files (#2032) @maxnoe
• Rework h5 metadata (#2003) @maxnoe
• Use pytest fixtures in unit tests instead of from_name methods (#1985) @maxnoe
• Store both equivalent and effective focal length in OpticsDescription, add __slots__ (#1923) @maxnoe
• Replace deepcopying defaults with default_factory (#1991) @maxnoe
• Use module of Tool to setup logging (#1964) @maxnoe
• Add QualityQuery.get_table_mask, compile python code (#1943) @maxnoe
• Load all dl2 available groups in TableLoader (#1934) @maxnoe
• Use container classes for TableReader.read method, do not reuse same container instance (#1921) @maxnoe

Data Model Changes

• ObsConfig data model (#1973) @kosack
• Rename ReconstructedContainer.tel_ids -> telescopes, fixes 1953 (#2044) @maxnoe
• Use n_ prefix consistently, (rename all num_ to n_), fixes #986 (#2046) @maxnoe
• Add pixel, channel, sample information to CameraReadout, update OpticsDescription (#2019) @maxnoe
• Rework h5 metadata (#2003) @maxnoe
• Fix datalevels (#2004) @maxnoe
• Store both equivalent and effective focal length in OpticsDescription, add __slots__ (#1923) @maxnoe
• Rework container prefix handling (#1949) @maxnoe
• Add location of subarray reference point to SubarrayDescription (#1980) @maxnoe
• compute impact distance to shower axis (#1899) @kosack

Bug Fixes

• Fix missing f-string (#2043) @nbiederbeck
• Fix columns order when string column present, fixes #2041 (#2042) @maxnoe
• Fix writing and merging of dl2 statistics, fixes #1884 (#1885) @maxnoe
• Fix pixel rotation for very large offsets in NominalFrame (#2037) @maxnoe
• Fix wrong pixel area and rotation in nominal frame, fixes #2028 (#2029) @maxnoe
• Fix error message for missing effective focal length (#2026) @maxnoe
• Remove restriction on existing file in EventSource.input_url, fixes #1821 (#1998) @maxnoe
• Fix TableLoader reading empty table (#2013) @nbiederbeck
• Fix datalevels (#2004) @maxnoe
• Include number of mirror tiles in guessing keys to be able to distinguish between MAGIC 1 and MAGIC 2 (#1977) @maxnoe
• Use correct traitlet for event type enum in eventtypefilter (#2002) @maxnoe
• Add dl2 datalevel in hdf5 source, add datalevel tests (#2001) @maxnoe
• Ignore unknown column metadata in read_table (#1992) @maxnoe
• Fix comparison of timestamps in calibration code (#1987) @maxnoe
• Fix missing start/stop for reading impact table, use correct type in simteleventsource (#1982) @maxnoe
• Hash rounded versions of floats for CameraGeometry (#1956) @maxnoe
• Fix impact being written for non-triggered telescopes, fixes #1961 (#1967) @maxnoe
• fixed bug that prevented printing of QualityQuery (#1958) @kosack
• Fix ArrayDisplay creating legend in wrong axes with multiple axes active (#1946) @maxnoe
• Store true_image_sum also when true image is not written (#1893) @maxnoe
• Better error message in case unknown telescopes are selected, fixes #1924 (#1925) @maxnoe
• Minor visual bugfix: better html repr for Tool and Component (#1905) @kosack

Refactoring and Optimization

• Use tel_id consistently (replacing telid -> tel_id), fixes #1457 (#2045) @maxnoe
• Remove cutflow (#1766) @maxnoe
• Add 'raises' to run_tool, which raises exceptions instead of logging them. (#2038) @nbiederbeck
• ExpressionEngine for QualityQuery and FeatureGenerator (#2014) @nbiederbeck
• Return tuples from SubArray properties, add suport for single value in get_tel_ids (#2027) @maxnoe
• Use pytest fixtures in unit tests instead of from_name methods (#1985) @maxnoe
• Ignore containers and maps by default in HDF5Writer (#1945) @maxnoe
• Hdf5EventSource performance improvements and skip empty events, fixes #1916 (#1917) @maxnoe
• Directly return peak position from neighbor average waveform (#1914) @maxnoe

Maintenance

• updated incorrect comments in base_config.yaml (#2040) @kosack
• Remove support for by_type split files (#2032) @maxnoe
• Fixed Typo in docstring (#2009) @aknierim
• added a better template for release-drafter (#1880) @kosack
• Add flake8 pre-commit hook. Change black hook to also format notebooks. (#1993) @nbiederbeck
• Avoid warning in case of reading DL1a only files with HDF5EventSource (#1994) @maxnoe
• Automatically include fields in docstrings for containers (#1990) @maxnoe
• Create a fresh ArrayEventContainer for each event in HDF5EventSource (#1970) @maxnoe
• Create a fresh ArrayEventContainer for each event in SimTelEventSource (#1952) @maxnoe
• Update CI actions versions (#1951) @maxnoe
• Simplify units code in subarray (#1950) @maxnoe
• Use a simple kwarg instead of a traitlet for TableLoader.h5file (#1937) @maxnoe
• Improve TableLoader docs, fixes #1930 (#1931) @maxnoe
• Print version info in pytest header (#1919) @maxnoe

v0.15.0 – 2022-05-23

What's Changed since v0.14.0

The main change in this release is that the SimTelEventSource will use the effective focal length,
which is the focal length corrected for the mean effect of aberrations like coma, by default.
This will raise an error if this information is not available in the input file (which is the case for files from before prod5).

To analyze these files with ctapipe 0.15, you need to pass the SimTelEventSource.focal_length_choice="nominal".
Using the effective focal length much improves the off-axis angular resolution of e.g. the HillasReconstructor.

• Use effective focal length by default, always guess telescope names from nominal (#1910) @maxnoe
• Fix ctapipe license in tutorial (#1907) @maxnoe
• Fix merge tool in case --skip-images is given but no images in input file (#1906) @maxnoe
• Keep z coordinate in tilted ground frame (#1903) @maxnoe
• Fix HDF5EventSource.len to take max_events into account (#1897) @maxnoe
• Add alias for prov log (#1890) @maxnoe
• update README for v0.14.0 (#1886) @kosack
• Fix setting up logging hierarchy, fixes #1882 (#1883) @maxnoe

Contributors

@kosack and @maxnoe

v0.14.0 – 2022-04-14

What's Changed since v0.13.0

This release fixes an important issue in the stereo reconstruction: the quality criteria for individual telescope events to enter into the stereo reconstruction were not applied correctly. Fixing this required an API change.
The quality criteria are now part of the Reconstructor as StereoQualityQuery, not part of the ShowerProcessor any more. As part of this change, we also added back support for using the HillasIntersection additionally to the HillasReconstructor.

The second large change is the addition of support for two better formats for config files. Additionally to the json supported before, ctapipe now also supports yaml and toml configuration files and tools support giving multiple configuration files.

• Fix bug in ShowerProcessor / HillasReconstructor not using QualityQuery correctly, add support for HillasIntersection in ShowerProcessor (#1875) @maxnoe
• allow YAML and TOML config files and multiple configs (#1856) @kosack
• Remove parallel=True for neighbor_waveform_sum, fixes #1532 (#1878) @maxnoe
• Add switch to ImageProcessor to enable/disable ImageModifier (#1873) @maxnoe

Contributors

@kosack and @maxnoe

v0.13.0 – 2022-04-08

This is mainly a maintenance release updating the versions of some core depedencies (python >= 3.8, astropy >= 5.0) but there are also some changes to the data model and format and a couple of new features and bug fixes.

What's Changed since v0.12.0

• Components to modify dl1 images (#1723) @LukasNickel
• Compute and record shower's core position also in the TiltedFrame when using HillasReconstructor (#1745) @HealthyPear
• Extractor status flag (#1810) @kosack
• Unify source.simulation_config return value (#1872) @LukasNickel
• Update black version (#1871) @LukasNickel
• Only upload codecov from one build (#1869) @maxnoe
• Fix traitlets deprecation warning (#1866) @maxnoe
• Implement public interface for 2d index to/from id (#1854) @maxnoe
• Implement merging dl2 subarray data, fixes #1861 (#1864) @maxnoe
• Enable Subarray.to_hdf/from_hdf for already opened file (#1868) @maxnoe
• Update CI os/install-method/python-version combinations (#1865) @maxnoe
• Use write_table for image statistics (#1863) @maxnoe
• Close event source in ProcessorTool.finish (#1862) @maxnoe
• Update bokeh viewers to version 2.0, remove bokeh viewer tool and WaveformDisplay for now (#1494) @maxnoe
• Add write_table and use it in SubarrayDescription.to_hdf, fixes #1449 (#1817) @maxnoe
• resolve environment variables in Path traitlets (#1855) @kosack
• Require python 3.8, astropy 5 (#1859) @maxnoe
• Do not run "push" build for Pull Requests, fix mamba build in CI (#1858) @maxnoe
• Fix CameraFrame transformations for arrays of focal_length, fixes #1850 (#1851) @maxnoe
• Add macos-latest to CI workflow jobs (#1718) @HealthyPear
• Add event type filter to ctapipe-process (#1841) @maxnoe
• Fix scipy deprecation warning (#1846) @maxnoe
• Point to readthedocs in the README (#1842) @maxnoe
• Implement treatment of invalid values in FixedPointColumnTransform, fixes #1815 (#1816) @maxnoe
• Just use a for loop in HDF5TableReader (#1845) @maxnoe
• Remove print statements from hdf5eventsource (#1844) @maxnoe
• Add RTD config (#1840) @maxnoe
• Fix _dev_version being included in wheel (#1839) @maxnoe
• Raise error for missing focal length (#1835) @maxnoe
• Rename DL1EventSource to HDF5EventSource, add support for reading R1 waveforms as written by DataWriter (#1837) @maxnoe
• show Frame in CameraGeometry repr (#1834) @kosack
• Fix pixel picker for square pixels (#1833) @maxnoe
• Use .ecsv extension in dump instrument to enable Table.read. (#1722) @maxnoe
• Add BrightestPixelsPeakWindowSum extractor (#1823) @maxnoe
• Remove outdated tools (#1827) @maxnoe
• Fix incorrect usage of configuration system in calibration calculators and tool example (#1831) @maxnoe
• Make tests pass independent of cwd (#1818) @maxnoe
• Read true_image_sum, fixes #1180 (#1814) @maxnoe
• options to write R0 and R1 waveforms (#1799) @kosack
• Fix CameraGeometry.transform for cam_rotations != 0, fix pix_rotation for square pixels (#1819) @maxnoe
• Exclude coverage 6.3 in CI due to nedbat/coveragepy#1310 (#1830) @maxnoe
• add a simple method to read metadata as a dictionnary from an hdf5 file and allow writing metadata to any path (#1796) @vuillaut
• Create some issue templates (#1824) @kosack
• Make sure a fixed random seed is used for toymodel test (#1811) @maxnoe

Contributors

@HealthyPear, @LukasNickel, @kosack, @maxnoe and @vuillaut

v0.12.0 – 2021-11-23

What's Changed since v0.11.0

The main changes in this release are related to expanding the data model to include DL2 and the computation of DL2 parameters. This also required changing the units of some DL1 parameters from the CameraFrame to the TelescopeFrame to make them easier to compare between telescopes, which includes an overhaul of how the HillasReconstructor works (supporting both frames as input), and adds include a new Component ShowerProcessor that generates DL2(a) information. The data model was expanded to include DL2 and to support a more general InstrumentDescription serialization. DL1Writer was refactored into a general DataWriter that writes the full data model. TableLoader provides a user interface to load all events at once from ctapipe output files, joining them as needed, which is useful for benchmarking.

The ctapipe-stage1 tool was removed and replaced with the more general ctapipe-process, which can now handle any transformation from R0 to DL2a, depending on what configuration file it is given. A ctapipe-quickstarttool was added to generate some starting config files, and a ctapipe-fileinfo tool to show file header information for any files written by ctapipe.

New Features

• Add flag to use camera frame for ctapipe-process (#1790) @maxnoe
• TableLoader (#1771) @maxnoe
• Add flags to enforce recomputation of dl1/dl2 (#1776) @LukasNickel
• ctapipe-quickstart tool (#1774) @kosack
• Add condition option to read_table (#1769) @maxnoe
• Implement allow_none=False for Path trait, fixes #1759 (#1760) @maxnoe
• ctapipe-fileinfo (#1756) @kosack
• Configurable contact information for DataWriter (#1757) @maxnoe
• Generalize image converter for square pixels (#1728) @LukasNickel
• Implement ctapipe-process (#1726) @kosack
• Add support for calibration scale and shift in R1 (#1749) @HealthyPear
• Direction reconstruction (also) from Telescope Frame (#1408) @HealthyPear
• Add no-flags, add short aliases, make aliases more consistent (#1748) @nbiederbeck
• Adapt subarray for 'allowed_tels' option in merger_tool (#1736) @Hckjs
• Implement a shower processor (#1675) @HealthyPear
• DL2 data model and output (#1673) @kosack
• Allow regexp in table name for TableWriter.exclude() (#1717) @kosack
• Shortform aliases (#1713) @maxnoe
• implements add_column_transform_regexp feature (#1690) @kosack
• Use iminuit>=2 (#1714) @nbiederbeck

Breaking Changes

Both of these change the data model, but the code remains backward-compatible.

• Use indices for camera / optics lookup in from_hdf / to_hdf (#1426) @maxnoe
• Hillas parameters in telescope frame (optionally) (#1591) @LukasNickel
• ctapipe-stage1 refactored into ctapipe-process (#1726)

Performance Improvements

• Do not guess pixel area when transforming (#1794) @maxnoe
• Improve performance of calc_pixel_neighbors, fixes #1791 (#1792) @maxnoe
• Precompile table regex for column exclusion, fixes #1719 (#1720) @maxnoe

Bug Fixes and Refactoring

• Compare to numpy integer (#1801) @LukasNickel
• Fix negative vmin/vmax raising error with mpl3.5, fixes #1805 (#1807) @maxnoe
• Fix display dl1 (#1795) @maxnoe
• Fix writing FITS in fileinfo for unicode (#1789) @maxnoe
• Fix TelescopeParameter for traitlets 5.1 (#1784) @maxnoe
• Remove unnecessary is_simulation for ImageProcessor (#1773) @maxnoe
• Remove debug prints, fixes #1764 (#1765) @maxnoe
• Remove redundant test code in simtel event source test code (#1747) @maxnoe
• Change REF_WIDTH to REFWIDTH to fit into 8 characters for FITS, fixes #1452 (#1753) @maxnoe
• Rename function. Fix #1733. (#1734) @nbiederbeck
• Fix ctapipe-merge with --allowed-tels (#1712) @nbiederbeck
• Set errordef for taubin fit (#1716) @maxnoe

Documentation, Testing, and Build updates

a big change is that tests are now parallelizable, leading to faster results from the CI system.

• Removing unwanted quotes for code block in readme (#1808) @vuillaut
• Add note about PYTHONNOUSERSITE to README (#1806) @maxnoe
• Fix deprecated keys in setup.cfg (#1804) @maxnoe
• Fix several warnings during tests (#1787) @maxnoe
• Unit tests: fix mutated camera geometry fixtures (#1793) @maxnoe
• Build docs using two cores, fix name of step (#1781) @maxnoe
• Do not use sphinx action for building docs (#1780) @maxnoe
• include stage1_config.json in package (#1752) @nbiederbeck
• Switch on doctests in the CI (#1731) @maxnoe
• Add tests for download functions (#1763) @maxnoe
• Fix docs for Path (#1758) @maxnoe
• Use pypa/build for packaging (#1725) @maxnoe
• Cache the test data (#1746) @maxnoe
• Improve dl1 fixtures and fix test concurrency issues (#1741) @maxnoe
• Parallel tests (#1740) @maxnoe
• Pyflakes in CI (#1735) @maxnoe
• Add citing information for version 0.11 (#1710) @LukasNickel

Contributors

@Hckjs, @HealthyPear, @LukasNickel, @kosack, @maxnoe, @nbiederbeck and @vuillaut

v0.11.0

What's Changed since v0.10.5

This is a major upgrade due to the removal of some deprecated and internally unused modules and functions.
We also changed the default of CameraCalibrator.apply_waveform_timeshift and invalid configuration options will now raise an error,
so some configs might need updates.

One major change is that the event sources now will adapt the subarray and trigger info if allowed_tels is used.

Breaking changes

• Change default of CameraCalibrator.apply_waveform_timeshift to False (#1692) @maxnoe
• Consistent use of allowed_tels (#1697) @LukasNickel
• Raise error in CameraReadout.from_name if no corresponding file can be found (#1703) @maxnoe
• Remove unused io modules, fixes #1527 (#1693) @maxnoe
• Change definition of concentration_cog, fixes #1571 (#1572) @maxnoe
• Transform warning into error for wrong config (#1658) @maxnoe
• Remove charge resolution tools, fixes #1634 (#1646) @maxnoe

New features

• Store package versions in provenance (#1685) @maxnoe
• Store used simtel R1 calibration in event.monitoring (#1691) @maxnoe
• Add start, stop, step to read_table (#1684) @maxnoe
• Support http(s):// and dataset:// for path traitlet (#1638) @maxnoe
• Add brightest_island function (#1587) @maxnoe
• Allow specifying extractors per telescope, fixes #1660 (#1661) @maxnoe
• Use new random API (#1708) @maxnoe
• Pin sphinx to 3.5 for now (#1709) @maxnoe
• Allow appending of rows to existing tables in HDF5TableWriter, fixes #1655 (#1657) @maxnoe

Bug fixes / maintenance

• Fix bugs in TwoPassWindowSum and define better unit-test (#1698) @HealthyPear
• Update documentation Makefile (#1705) @HealthyPear
• Fix hillas intersection for "missing" telescope ids, fixes #1635 (#1702) @maxnoe
• Update mailmap (#1700) @maxnoe
• Fix two numpy deprecation warnings (#1704) @maxnoe
• Fix pixel picking in mpl display, add test (#1701) @maxnoe
• Fix dl1source (#1696) @LukasNickel
• Make sure hdf5 file is closed when necessary in read_table, fixes #1694 (#1695) @maxnoe
• Treat sphinx warnings as erros, fix deploy workflow name (#1686) @maxnoe
• Do not use numba cache for lts_linear_fit (#1680) @maxnoe
• Fix syntax typo in call of TwoPassWindowSum with 2ndPass disabled (#1681) @HealthyPear
• Remove DL1Writer.setup, just properly initialize in init (#1667) @maxnoe
• Fix codecov missing coverage for cli tools (#1652) @maxnoe
• Use tqdm.auto (#1659) @maxnoe
• Require at least astropy 4.0.5, drop restriction to h5py 2.x (#1676) @maxnoe
• Fix some documentation warnings (#1677) @HealthyPear
• Create cache dir only when needed (#1670) @maxnoe
• Use conda-forge channel in README and add some notes (#1671) @maxnoe
• Update dl1writer.py docstring (#1666) @vuillaut
• Fix time_delta cleaning (#1656) @LukasNickel
• Update badge to 0.10.5 (#1650) @maxnoe
• Generalize URL for downloading test data (#1651) @HealthyPear
• Update README.rst (#1647) @orelgueta

Contributors

@HealthyPear, @LukasNickel, @maxnoe, @orelgueta and @vuillaut

v0.10.5

What's Changed since v0.10.4

• fix a bug in twopass extractor where the timing slope was wrong (#1639) @kosack
• Add missing f-string in dl1-merge to see broken filenames, fixes #1643 (#1644) @Hckjs
• Make sure each test gets a copy of event/subarray (#1637) @maxnoe

Contributors

@Hckjs, @kosack and @maxnoe