Speedup muon validation #33610
Speedup muon validation #33610
Conversation
|
-code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33610/22451
Code check has found code style and quality issues which could be resolved by applying following patch(s)
|
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33610/22495
|
|
A new Pull Request was created by @abbiendi for master. It involves the following packages: SimMuon/MCTruth @civanch, @kmaeshima, @andrius-k, @mdhildreth, @ErnestaP, @cmsbuild, @jfernan2, @ahmad3213, @rvenditti can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
please test |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b486f6/14877/summary.html CMS Clang-Tidy warnings: There are Clang-Tidy warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b486f6/14877/llvm-analysis/cmsclangtidy.txt for details. Comparison SummarySummary:
|
|
@abbiendi I don't understand why the PR is introducing changes also in Tracking and Vertexing for two wfs apart from those expected in Muons/recoMuonV All the changes here: Thanks |
|
@jfernan2 please can you tell me how can I access the DQM files directly ? I tried to download one with "wget" but it failed. Browsing with firefox I am not sure what I see. Some histos are missing ? I suspect the compared files could have different statistics |
|
@abbiendi you can access the full DQM root files, not just the differences as I pointed you out, on the following folders: |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b486f6/15283/summary.html CMS Clang-Tidy warnings: There are Clang-Tidy warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b486f6/15283/llvm-analysis/cmsclangtidy.txt for details. Comparison SummarySummary:
|
|
Please, if you want me to repeat my test, give me the recipe, I am a bit lost... |
@abbiendi you can try to compare manually |
done! |
|
@abbiendi thank you for the check! |
|
please test |
|
#33803 is merged |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b486f6/15404/summary.html CMS Clang-Tidy warnings: There are Clang-Tidy warnings. See https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b486f6/15404/llvm-analysis/cmsclangtidy.txt for details. Comparison SummarySummary:
|
|
+1 |
|
+1 |
|
With this PR merged, we got IB failure with workflow 1311.0: @abbiendi would you please have a fix soon? With last IB CMSSW_12_0_X_2021-05-30-0000, it works well. |
|
I'm looking into it |
|
The same problem I have in a few TSG tests! |
|
it's a bit tricky, not yet understood. The broken workflow seems to have a "fake" hlt, and the muon track collections are not there in the Event. So there is nothing to do for the validation. I do not see why the new IB is broken while until CMSSW_12_0_X_2021-05-30-0000 everything worked. The muon hlt collections were absent also there. |
|
I think all three relvals failing started after this (at lest it shows the same missing products) |
|
I meant I see the same problem, ie, same error message! A collection being absent is not allowed to crash the workflow esp. for HLT as the trigger producing the collection may not have been run to completion, so missing products must be expected. So this behaviour seems to have been changed by this PR. |
|
maybe I found the problem |
|
should be ok with PR#33938 |
|
Indeed solves the problem in the TSG tests, thanks! |
This PR is mostly aimed at reducing the cpu time needed for the official muon validation sequence.
It restricts the SimToReco association to subsamples of the TrackingParticle (TP) collection, relevant for the muon validation, which are preselected as vectors of references in the Event.
"probeTracks", i.e. inner tracks kinematically compatible with identified muons, as well as "displacedTracks", are tested with respect to TP with ptmin > 2 GeV
Particle Flow muons, are tested w.r.t. general TP with ptmin>0.5 GeV && ptot>2.5 GeV (without muon truth selection)
All the other muon collections (including the hlt muon collections) are tested w.r.t. :
These definitions seem conservative, could be fine-tuned or the logic could be modified if needed.
In addition a fix is applied to the validation of global muons which now by default requires the track to have at least a valid muon hit (therefore global muons made of only inner tracker hits are considered as fakes). A new parameter in MuonAssociatorByHits is controlling this behaviour (called rejectBadGlobal, default is True. In a few cases it is set to False to allow tracks extending only in the inner tracker, according to the expected algorithm, as for DYT or TuneP).
The result of these changes is that a reconstructed muon can be considered as a fake not only if it does not pass the quality/purity thresholds, but also if there is no found TrackingParticle matching it in the selected subsample.
No changes are produced on the standalone (outer) muons.
(except that the denominator for efficiencies is restricted to the selected TP, so TP which will never reach the muon detectors are generally not counted for the efficiency. This gives a small increase of efficiencies throughout)
Almost no changes of the pure inner tracks (except that due to the better cpu performance I lowered the ptmin of probed tracks to 3 GeV, it was 4 GeV).
Increased fake rates for reco muons (where with this I mean all muon flavours where one expects a matching between a inner track and hits or segments, therefore including Tracker muons, GEM muons etc...), also depending on the chosen settings.
In practice for reco muons the fake rates quantify also the misid rate, not just the rate of imperfect tracking.
PR validation:
The results have been validated w.r.t. the default CMSSW_11_3_0_pre6 on about 1K events of:
TTbar+PU (mcRun3, 2021)
https://abbiendi.web.cern.ch/abbiendi/MuonVal_cpuTime/CMSSW_11_3_0_pre6_TPnewSel_vs_Default/RelValTTbar_14_PU_2021/
ZMM+PU (mcRun3, 2021)
https://abbiendi.web.cern.ch/abbiendi/MuonVal_cpuTime/CMSSW_11_3_0_pre6_TPnewSel_vs_Default/RelValZMM_14_PU_2021/
In addition basic tests have been done for a number of other workflows: SingleMuon gun, FastSim, Phase2, and premixing, all successful.
Obviously this PR should not enter in release together with significant changes to the muon reco or ID, otherwise a dedicated validation is needed to decouple the effects.