Integration Dispersion or Charge Resolution?

[watsonjj]

Is there a reason the "integration dispersion" is used as opposed to the Charge Resolution?

For those not aware: the Charge Resolution compares the measured (integrated + calibrated) charge to the true charge. Analogous to the Root-Mean-Square Error, the fractional Charge Resolution sigma_q/Q_T for a particular “true charge” Q_T (the number of photoelectrons that were produced in the sensor, before multiplication) is defined as:

where N is the total number of measured charges, Q_M_i , with that value of Q_T. It therefore takes into account both the bias and the spread of the measured charge about the "true charge".

When using the photo_electron_image from the simtelarray files as Q_T, the Poisson fluctuations in photoelectron number need to be included into the equation:

This ensures that a perfect detector which consistently reads-out a “measured charge” with an equal value to the “true charge” would still hit the Poisson limit. This limit ensures realistic conclusions can be reached from the Monte Carlo simulations, as it is not physically possible to know the “true charge” generated inside the photosensor, free from fluctuations.

An example of a Charge Resolution plot, comparing a result of a simulated camera at different values of NSB, is shown below:

In the past, the Charge Resolution had CTA Requirement attached to it, however it has since been updated/replaced with the "Intensity Resolution", where the measured signal in photons is compared to the true signal in photons. This change of definition from photoelectrons to photons was performed on the majority of the requirements, as it is a more coherent representation of performance in observing Cherenkov showers, especially with the variety of cameras and photosensors in CTA.

To expand on that, in the case of SiPMs it is possible to reduce the bias voltage applied to reduce the Excess Noise Factor, therefore improving the Charge Resolution. However, this also reduces the Photon Detection Efficiency, which is also important to consider. This is correctly reflected in the Intensity Resolution, therefore making it a better requirement than Charge Resolution.

The definition of the Intensity Resolution can be found on Jama:

As explained, the full proof of a camera meeting this requirement is performed with MC simulations of Cherenkov showers, the Charge/Intensity Resolution of the real camera in the lab, and the proof that the model matches reality with the comparison against the MC simulation of the camera in the lab. It is also important to note that the requirement must be met under specified levels of NSB.

How we should obtain Intensity Resolution from MC is currently unclear - the true number of photons is not stored by default in the simtelarray file. One approximation is to multiply the number of photoelectrons by the Quantum Efficiency/Photon Detection Efficiency.

For the time being, the Charge Resolution can be calculated in ctapipe using the ctapipe.analysis.camera.charge_resolution.ChargeResolutionCalculator class. A script which uses this class is located in ctapipe/tools/extract_charge_resolution.py. It produces a HDF5 file containing the Charge Resolution at each true charge.

There was a suggestion in cta-observatory/ctapipe/pull/857 to create a extract_dl1 Tool, which creates a file containing the DL1 containers, which then can be read into scripts like extract_charge_resolution.py. This will simplify the script and not require it to loop over and calibrate events itself.

The class ctapipe.plotting.charge_resolution.ChargeResolutionPlotter can plot multiple Charge Resolution results onto the same axis. It also contains the equations for the old Charge Resolution requirements, which can also be plot onto the axis. A script which uses this class can be found at ctapipe/tools/plot_charge_resolution.py, which can take a list of files produced from extract_charge_resolution, and plot them on the same axis.

There are many factors which contribute to the Charge/Intensity resolution of a camera. It encapsulates the electronics performance, the waveform calibration (R0->DL0), the signal extraction approach, and the signal calibration. Additionally, the answer to "which charge integration is best for my camera?" can also depend on "what NSB are you observing at?" and "are you illuminating the camera with Cherenkov Shower ellipses or a uniform illumination?". It is therefore appropriate to use the Charge/Intensity Resolution as a benchmark for the charge extraction, as it has associated requirements that the camera must meet anyway, of which define the conditions the benchmark must be assessed under.

[vuillaut]

Hi @watsonjj

Is there a reason the "integration dispersion" is used as opposed to the Charge Resolution?

The only reason there was my lack of knowledge.
I wanted to assess the calibration methods and ended up with this benchmark.
Thank you very much for bringing all that information forward.

[watsonjj]

Sorry for the information overload. I thought it would be useful to include it in this repository, as it can be hard to find specifics about some of these procedures. This repository seems like the appropriate place for Charge Resolution to be assessed, ensuring a common procedure is undertaken for all telescopes.

Perhaps we may also consider moving the charge resolution code from ctapipe to this repository in the future.

[vuillaut]

Better have all the info :)
Maybe it would be good also to add some of it in the benchmark document?

[moralejo]

I would like to add a point: in the current integration_dispersion_ipynb the comparison of reconstructed and true number of photoelectrons is done on the sum of all charges in pixels which contain at least one (true) p.e. It should be done on a pixel-wise basis, since the information (in the simtel file that is used as input currently) is there. I tried that in a simple way (not using ChargeResolutionCalculator), see here:
https://github.com/moralejo/cta-benchmarks/blob/master/Benchmarks/DL0/charge_resolution.ipynb

It is a bit worrying that for LST there is a population of events with true charges up to 50 p.e. and ~0 reconstructed charge. It may be simply pulses which fell out of the readout window, but would be good to make sure. One can also see saturation of the high gain - we should include the low gain in the benchmark, obviously. In the current benchmark (adding all p.e.'s in an event) all these effects do mix up in each event, and hence makes the result hard to interpret.
Finally, the benchmark should allow for some variation of the pulse integrator parameters, e.g. number of integrated samples and how the pulse peak is obtained. By default, is it just the highest sample, or the maximum from a "sliding integration window" of a few samples? the latter should be more robust for small signals.