Load CLCT pre-trigger digis in CSC packer. Skeleton code to pack/unpack CSCShowerDigi (HadronicShowerTrigger-3) #33444

dildick · 2021-04-15T17:46:54Z

PR description:

This PR loads CLCT pre-trigger digis (CSCCLCTPreTriggerDigiCollection) in packer, in preparation of packing the CSCStripDigi based on the pretrigger decision (as is done in CMS during data taking). In addition, skeleton code was added to pack/unpack CSCShowerDigi.

Neither addition modifies the CSC packer/unpacker.

I also cleaned up the configuration of cscpacker in cscPacker_cfi. The same definition is already in the C++ module (fillDescriptions) and then again as commented Python code in the same file. I removed the duplicates. So cscpacker is now defined as a clone of cscPackerDef which is automatically generated by the compiler.

PR validation:

Code compiles and runs with WF 11634.0. There should not be any changes in any workflows.

11634.0_TTbar_14TeV+2021+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED Step4-PASSED  - time date Thu Apr 15 12:32:46 2021-date Thu Apr 15 12:03:53 2021; exit: 0 0 0 0 0
1 1 1 1 1 tests passed, 0 0 0 0 0 failed

if this PR is a backport please specify the original PR and why you need to backport that PR:

N/A

Before submitting your pull requests, make sure you followed this checklist:

verify that the PR is really intended for the chosen branch
verify that changes follow CMS Naming, Coding, And Style Rules
verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

cmsbuild · 2021-04-15T17:54:27Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33444/22129

This PR adds an extra 48KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File EventFilter/CSCRawToDigi/interface/CSCEventData.h modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429
- File EventFilter/CSCRawToDigi/interface/CSCVTMBHeaderFormat.h modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429
- File EventFilter/CSCRawToDigi/plugins/CSCDCCUnpacker.cc modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429
- File EventFilter/CSCRawToDigi/src/CSCDigiToRaw.cc modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429
- File EventFilter/CSCRawToDigi/src/CSCEventData.cc modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429

cmsbuild · 2021-04-15T17:54:52Z

A new Pull Request was created by @dildick (Sven Dildick) for master.

It involves the following packages:

EventFilter/CSCRawToDigi

@perrotta, @jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @ptcox, @barvic this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

perrotta · 2021-04-16T06:02:52Z

please test

cmsbuild · 2021-04-16T07:26:50Z

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d9cba6/14264/summary.html
COMMIT: 6e58819
CMSSW: CMSSW_12_0_X_2021-04-15-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/33444/14264/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

8.0

----- Begin Fatal Exception 16-Apr-2021 08:27:48 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLTAnalyzerEndpath'
   [2] Prefetching for module L1GlobalTriggerRawToDigi/'hltGtDigis'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Calling method for module CSCDigiToRawModule/'cscpacker'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: MuonDigiCollection<CSCDetId,CSCCLCTPreTriggerDigi>
Looking for module label: simCscTriggerPrimitiveDigis
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

5.1

----- Begin Fatal Exception 16-Apr-2021 08:28:21 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'FEVTDEBUGHLToutput_step'
   [2] Prefetching for module PoolOutputModule/'FEVTDEBUGHLToutput'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Calling method for module CSCDigiToRawModule/'cscpacker'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: MuonDigiCollection<CSCDetId,CSCCLCTPreTriggerDigi>
Looking for module label: simCscTriggerPrimitiveDigis
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

9.0

----- Begin Fatal Exception 16-Apr-2021 08:30:59 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'HLTAnalyzerEndpath'
   [2] Prefetching for module L1GlobalTriggerRawToDigi/'hltGtDigis'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Calling method for module CSCDigiToRawModule/'cscpacker'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: MuonDigiCollection<CSCDetId,CSCCLCTPreTriggerDigi>
Looking for module label: simCscTriggerPrimitiveDigis
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

25.0

RelVals-INPUT

8.08.0_BeamHalo+BeamHaloINPUT+DIGICOS+RECOCOS+ALCABH+HARVESTCOS/step2_BeamHalo+BeamHaloINPUT+DIGICOS+RECOCOS+ALCABH+HARVESTCOS.log
9.09.0_Higgs200ChargedTaus+Higgs200ChargedTausINPUT+DIGI+RECO+HARVEST/step2_Higgs200ChargedTaus+Higgs200ChargedTausINPUT+DIGI+RECO+HARVEST.log
6.06.0_SingleMuPt1+SingleMuPt1INPUT+DIGI+RECO+HARVEST/step2_SingleMuPt1+SingleMuPt1INPUT+DIGI+RECO+HARVEST.log

Expand to see more relval errors ...

AddOn Tests

hlt_mc_Fake

----- Begin Fatal Exception 16-Apr-2021 08:28:35 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 3 stream: 1
   [1] Running path 'RAWSIMoutput_step'
   [2] Prefetching for module PoolOutputModule/'RAWSIMoutput'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Calling method for module CSCDigiToRawModule/'cscpacker'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: MuonDigiCollection<CSCDetId,CSCCLCTPreTriggerDigi>
Looking for module label: simCscTriggerPrimitiveDigis
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

dildick · 2021-04-16T12:41:53Z

I see. It's failing for Run-1 scenarios where the pre-trigger based packing was enabled. I just need to always write out the pre-trigger digis in the trigger, and then it should be fine for all workflows.

cmsbuild · 2021-04-16T14:17:28Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33444/22143

This PR adds an extra 48KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File EventFilter/CSCRawToDigi/interface/CSCEventData.h modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429
- File EventFilter/CSCRawToDigi/interface/CSCVTMBHeaderFormat.h modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429
- File EventFilter/CSCRawToDigi/plugins/CSCDCCUnpacker.cc modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429
- File EventFilter/CSCRawToDigi/src/CSCDigiToRaw.cc modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429
- File EventFilter/CSCRawToDigi/src/CSCEventData.cc modified in PR(s): Use CSCConstants in CSC trigger, packer, unpacker and track-finders #33429
- File L1Trigger/CSCTriggerPrimitives/python/cscTriggerPrimitiveDigis_cfi.py modified in PR(s): Hit counters for Run-3 shower CSC trigger (HadronicShowerTrigger-2) #33416

cmsbuild · 2021-04-16T14:18:00Z

Pull request #33444 was updated. @perrotta, @cmsbuild, @rekovic, @slava77, @jpata, @cecilecaillol can you please check and sign again.

dildick · 2021-04-16T14:19:13Z

This runs WF 8.0 now

8.0_BeamHalo+BeamHalo+DIGICOS+RECOCOS+ALCABH+HARVESTCOS Step0-PASSED Step1-PASSED Step2-PASSED Step3-PASSED Step4-PASSED  - time date Fri Apr 16 09:03:41 2021-date Fri Apr 16 08:44:56 2021; exit: 0 0 0 0 0
1 1 1 1 1 tests passed, 0 0 0 0 0 failed

dildick · 2021-04-16T14:21:52Z

Available collections in Run-1 (re-)emulation are

CSCDetIdCSCCorrelatedLCTDigiMuonDigiCollection_simCscTriggerPrimitiveDigis_MPCSORTED_HLT. 566 373.2
CSCDetIdCSCCorrelatedLCTDigiMuonDigiCollection_simCscTriggerPrimitiveDigis__HLT. 553.4 360.3
CSCDetIdCSCALCTPreTriggerDigiMuonDigiCollection_simCscTriggerPrimitiveDigis__HLT. 871 337.5
CSCDetIdCSCCLCTPreTriggerDigiMuonDigiCollection_simCscTriggerPrimitiveDigis__HLT. 397.8 336
CSCDetIdCSCALCTDigiMuonDigiCollection_simCscTriggerPrimitiveDigis__HLT. 399 317.2
CSCDetIdCSCCLCTDigiMuonDigiCollection_simCscTriggerPrimitiveDigis__HLT. 358.2 311.7
CSCDetIdintMuonDigiCollection_simCscTriggerPrimitiveDigis__HLT. 326.4 290.8

dildick · 2021-04-16T15:53:53Z

This PR should be orthogonal to #33429, so it can go in before or after.

cmsbuild · 2021-04-26T13:44:22Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33444/22290

This PR adds an extra 28KB to repository

cmsbuild · 2021-04-26T13:44:43Z

Pull request #33444 was updated. @perrotta, @cmsbuild, @rekovic, @slava77, @jpata, @cecilecaillol can you please check and sign again.

slava77 · 2021-04-26T13:44:48Z

@cmsbuild please test

cmsbuild · 2021-04-26T16:21:34Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d9cba6/14580/summary.html
COMMIT: dce3bc4
CMSSW: CMSSW_12_0_X_2021-04-26-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/33444/14580/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2877605
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2877582
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 37 files compared)
Checked 160 log files, 37 edm output root files, 38 DQM output files
TriggerResults: no differences found

cecilecaillol · 2021-04-27T07:32:22Z

+l1

slava77 · 2021-04-27T15:21:20Z

@dildick
please add a link to a presentation in a DPG meeting (if any).

@ptcox
please clarify if the changes look OK for CSC DPG.
Thank you.

dildick · 2021-04-27T15:42:21Z

Hi Slava, I don't have a presentation for this part yet. I need to do a few studies on MC simulation with the CSCCLCTPreTriggerDigi object. But basically, if remember the readout condition for strips in a (D)CFEB correctly it's a L1-accept with a pre-trigger CLCT in that (D)CFEB. I would have to check the CFEB of each strip and then check if there is CSCCLCTPreTriggerDigi in that corresponding (D)CFEB. That would have to be done in this function https://github.com/cms-sw/cmssw/blob/master/EventFilter/CSCRawToDigi/src/CSCDigiToRaw.cc#L73-L101. It seems that bool accept(const CSCDetId& cscId, const CSCCLCTPreTriggerCollection& lcts, int bxMin, int bxMax, bool me1abCheck) does a BX check for the whole chamber, but not per (D)CFEB.

ptcox · 2021-04-27T16:33:38Z

Hi Slava, Sven, As far as I can tell Sven is doing the right thing here. We DO want to activate the readout suppression modelling in simulation again (e.g. EXO-20-015 had to do some extra work to check the lack of this suppression was not disturbing their simulation predictions), and it is the preLCT (pre-CLCT) trigger primitive which is required to start storage of strip pulse heights. I agree that in the hardware the granularity is by CFEB (which handles 16 strips x the 6 layers of a CSC) for readout, but I think we first considered working per chamber would be a pretty good approximation and that's why Rick Wilkinson's old 'accept' functions work per CSC rather than per CFEB. For now, I'd say get things working per chamber and then later consider if it's necessary to go to per-CFEB. Regards, Tim Slava Krutelyov wrote on 4/27/21 17:21:

…

@dildick <https://github.com/dildick> please add a link to a presentation in a DPG meeting (if any). @ptcox <https://github.com/ptcox> please clarify if the changes look OK for CSC DPG. Thank you. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#33444 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGYLHQ7OND6EEOI6EAZEDDTK3JARANCNFSM4276USRA>.

slava77 · 2021-04-27T16:39:49Z

I would have to check the CFEB of each strip and then check if there is CSCCLCTPreTriggerDigi in that corresponding (D)CFEB.

Sven,
I'm a bit confused; does this mean that some changes or checks still need to be done in this PR from your side?
Please clarify.

dildick · 2021-04-27T16:44:42Z

@slava77 No, this is good enough for now. The code is now able to load the pre-trigger digis and CSC showers digis. In a follow-up (to be presented at CSC DPG) I will implement the actual feature to pack with the CSCCLCTPreTriggerDigi.

slava77 · 2021-04-27T16:58:07Z

+reconstruction

for #33444 dce3bc4

code changes are in line with the PR description and the follow up review
jenkins tests pass and comparisons with the baseline show no differences in the monitored variables; there is an addition of 4 products in the DIGI step of the workflows

 CSCDetIdCSCALCTPreTriggerDigiMuonDigiCollection_simCscTriggerPrimitiveDigis__*.
 CSCDetIdCSCCLCTPreTriggerDigiMuonDigiCollection_simCscTriggerPrimitiveDigis__*.
 CSCDetIdCSCShowerDigiMuonDigiCollection_simCscTriggerPrimitiveDigis_Anode_*.
 CSCDetIdCSCShowerDigiMuonDigiCollection_simCscTriggerPrimitiveDigis__*.

cmsbuild · 2021-04-27T16:58:33Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2021-04-28T07:40:57Z

+1

cmsbuild added this to the CMSSW_12_0_X milestone Apr 15, 2021

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Apr 15, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 15, 2021

dildick changed the title ~~Load CLCT pre-trigger digis in CSC packer. Skeleton code to pack/unpack CSCShowerDigi~~ Load CLCT pre-trigger digis in CSC packer. Skeleton code to pack/unpack CSCShowerDigi (HadronicShowerTrigger-3) Apr 15, 2021

cmsbuild added tests-started and removed tests-pending labels Apr 16, 2021

cmsbuild added tests-rejected and removed tests-started labels Apr 16, 2021

cmsbuild added code-checks-pending l1-pending tests-pending and removed code-checks-approved tests-rejected labels Apr 16, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 16, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 26, 2021

cmsbuild added tests-started and removed tests-pending labels Apr 26, 2021

cmsbuild added tests-approved and removed tests-started labels Apr 26, 2021

cmsbuild added l1-approved and removed l1-pending labels Apr 27, 2021

cmsbuild removed pending-signatures reconstruction-pending labels Apr 27, 2021

cmsbuild added fully-signed reconstruction-approved labels Apr 27, 2021

cmsbuild added orp-approved and removed orp-pending labels Apr 28, 2021

cmsbuild merged commit 9f77c3d into cms-sw:master Apr 28, 2021

dildick deleted the from-CMSSW_11_3_X_2021-04-15-1100-pack-cscshowerdigi-v3 branch April 28, 2021 11:43

dildick mentioned this pull request Nov 6, 2021

uGT framework and L1 ntuples for showers in the muon system for Run-3 (HadronicShowerTrigger-8) #36018

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Load CLCT pre-trigger digis in CSC packer. Skeleton code to pack/unpack CSCShowerDigi (HadronicShowerTrigger-3) #33444

Load CLCT pre-trigger digis in CSC packer. Skeleton code to pack/unpack CSCShowerDigi (HadronicShowerTrigger-3) #33444

dildick commented Apr 15, 2021

cmsbuild commented Apr 15, 2021

cmsbuild commented Apr 15, 2021

perrotta commented Apr 16, 2021

cmsbuild commented Apr 16, 2021

dildick commented Apr 16, 2021

cmsbuild commented Apr 16, 2021

cmsbuild commented Apr 16, 2021

dildick commented Apr 16, 2021

dildick commented Apr 16, 2021

dildick commented Apr 16, 2021

cmsbuild commented Apr 26, 2021

cmsbuild commented Apr 26, 2021

slava77 commented Apr 26, 2021

cmsbuild commented Apr 26, 2021

cecilecaillol commented Apr 27, 2021

slava77 commented Apr 27, 2021

dildick commented Apr 27, 2021

ptcox commented Apr 27, 2021 via email

slava77 commented Apr 27, 2021

dildick commented Apr 27, 2021

slava77 commented Apr 27, 2021

cmsbuild commented Apr 27, 2021

silviodonato commented Apr 28, 2021

Load CLCT pre-trigger digis in CSC packer. Skeleton code to pack/unpack CSCShowerDigi (HadronicShowerTrigger-3) #33444

Load CLCT pre-trigger digis in CSC packer. Skeleton code to pack/unpack CSCShowerDigi (HadronicShowerTrigger-3) #33444

Conversation

dildick commented Apr 15, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Apr 15, 2021

cmsbuild commented Apr 15, 2021

perrotta commented Apr 16, 2021

cmsbuild commented Apr 16, 2021

RelVals

RelVals-INPUT

AddOn Tests

dildick commented Apr 16, 2021

cmsbuild commented Apr 16, 2021

cmsbuild commented Apr 16, 2021

dildick commented Apr 16, 2021

dildick commented Apr 16, 2021

dildick commented Apr 16, 2021

cmsbuild commented Apr 26, 2021

cmsbuild commented Apr 26, 2021

slava77 commented Apr 26, 2021

cmsbuild commented Apr 26, 2021

Comparison Summary

cecilecaillol commented Apr 27, 2021

slava77 commented Apr 27, 2021

dildick commented Apr 27, 2021

ptcox commented Apr 27, 2021 via email

slava77 commented Apr 27, 2021

dildick commented Apr 27, 2021

slava77 commented Apr 27, 2021

cmsbuild commented Apr 27, 2021

silviodonato commented Apr 28, 2021