Validation suite

[gipert]

We should start planning for a serious validation suite, to be hopefully incorporated in our GitHub CI. The idea is to test all the microphysics in the simulation and compare to models/literature every time there is an upgrade.

Here are some simpler ideas that would already help (from a conversation with R. Henning):

• Design a simple geometry in remage like a HPGe detector in LAr with light readout and a variety of sources. When there is a software upgrade, we run high-stats simulations and generate diagnostic pdfs (e.g. energy spectra, heat maps, source spectra, etc.). We then compare these pdfs to those from previous runs and investigate differences. Of course this only finds differences and doesn’t check the physics itself, but should be easy to implement. We could also just use an existing geometry, but I think we’d want something that can run quickly and designed around testing physics we’re be concerned about.
• A more sophisticated version would be to compare to date from a well-characterized but simple test-stand. The challenge her would be to find a stand where the measurement systematics are small and well-understood.

[EricMEsch]

In examples/06-NeutronCapture I have contributed an IsotopeOutputScheme, that can be used in this validation suite to check isotope production and with that capture cross sections etc.

If we have some ideas on how exactly we want to run/design this validation suite i would be glad to add something like that.

[tdixon97]

Other things we could do, not neccesarily really validation.

• compare output to MaGe for a simple simulation
• could try some very simplified public datasets eg [g4-em-validation]
• for the test stand we probably need to use a variety of different ones, not sure how easily it can be included in CI, but we should start working on that soon

[gipert]

@EricMEsch if you have time, it would be great contribution. We can meet in the next weeks to define a list of tests to be implemented

[EricMEsch]

@EricMEsch if you have time, it would be great contribution. We can meet in the next weeks to define a list of tests to be implemented

Sure, I will be available next weeks

[gipert]

Note

Work in progress!

Unit tests

• test the LH5 output generation code, i.e. that we don't lose events on the way. This can be done by running a simulation with predictable number of events and check that we get all of them written out
• test whether setting different seeds generates a different statistical realization (especially with multithreading)

"Absolute" low-level validation

Physics list and Geant4 built-in generators

• print list of registered processes per particle
• production cuts: plot intensity of bremsstrahlung photons / low-angle Compton scatters vs cut value. Goal: understand if observable is stable and if we chose a good cut value
• check step sizes by plotting distribution of simulations from point above. Relevant for active volume so e.g. shoot a beta against a Ge surface
• unit tests of decay chains like Ra226 (long livetime threshold)
• cross check neutron physics with MCNP
• check Ar39 beta shape, maybe interesting to compare to bxdecay0

Generators

• confinement: option to force G4 containment check after vertex generation
• find way to run confinement tests in CI and make figures downloadable
• implement option to store track pre-tracking information: New output table to store track vertex information #178
• start plotting vertex data and identify useful plots for each kind of physics (double beta, cosmogenics, ...)
• check bxdecay0, is it implementing the best double-beta calculations?

"Relative" validation

Compare results with remage before the change.

High-level validation

• L200 calibration rates
• Simple test stand: HPGe and radioactive source at different distances

Compare to MaGe

• input simple GDML from high-level validation above and compare output (e.g. energy histograms)

Existing Geant4 validation:

https://geant4-internal.web.cern.ch/publications_validations/testing_and_validation

L200 geometry and materials

See legend-exp/legend-pygeom-l200#66

[gipert]

Found the right link to the Geant4 validation portal: https://geant-val-web.cern.ch/

[gipert]

We should try to validate the vertex confinement code more accurately. I've been simulating 0vbb in all detectors at the same time these days and I get weird results. Could be due to a bug in the way remage samples between different volumes.