Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EXO regional tracking iteration #32806

Closed
wants to merge 1 commit into from

Conversation

aehart
Copy link
Contributor

@aehart aehart commented Feb 3, 2021

PR description:

This PR introduces a new tracking iteration for Run 3, called the exoRegionalStep, based on ROIs targeting exotic physics scenarios. The ROIs are formed from tracks found by previous iterations, and they are tagged with an MVA that helps to reduce the CPU time. The tagged ROIs are used as tracking regions for the new iteration.

Several presentations have been made regarding the new iteration, most recently in the TRK POG meeting here:
https://indico.cern.ch/event/961478/#5-displaced-tracking-ml-develo

PR validation:

We see a useful increase in tracking efficiency for highly displaced tracks from at least two different exotic physics models (see above presentation for details). This is accompanied by a modest increase in the CPU time of just the tracking sequence of ~7%.

The usual code formatting checks have been also been run.

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 3, 2021

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32806/20986

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 3, 2021

A new Pull Request was created by @aehart (Andrew Hart) for master.

It involves the following packages:

DQM/TrackingMonitor
DQM/TrackingMonitorSource
DataFormats/GsfTrackReco
DataFormats/TrackReco
RecoParticleFlow/PFTracking
RecoTracker/ExoRegionalTracking
RecoTracker/FinalTrackSelectors
RecoTracker/IterativeTracking
RecoTracker/TkTrackingRegions
RecoVertex/V0Producer
Validation/RecoTrack

The following packages do not have a category, yet:

RecoTracker/ExoRegionalTracking
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@perrotta, @andrius-k, @kmaeshima, @ErnestaP, @ahmad3213, @cmsbuild, @jfernan2, @slava77, @jpata, @rvenditti can you please review it and eventually sign? Thanks.
@wmtford, @felicepantaleo, @jandrea, @fioriNTU, @threus, @seemasharmafnal, @mmarionncern, @hdelanno, @makortel, @JanFSchulte, @lgray, @sroychow, @GiacomoSguazzoni, @rovere, @VinInn, @hatakeyamak, @mschrode, @idebruyn, @ebrondol, @mtosi, @dgulhan, @arossi83, @lecriste, @cbernet, @gpetruc this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

@slava77
Copy link
Contributor

slava77 commented Feb 3, 2021

@cmsbuild please test

beamhalo = 28,
gsf = 29,
exoRegionalStep = 26,
reservedForUpgrades2 = 27,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when we discussed integration of what's now called displacedGeneralStep, the idea for the regional was that it can still reuse reservedForUpgrades2 and that the naming would be similar, displacedRegionalStep

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I renamed the iteration exoRegionalStepdisplacedRegionalStep and it now occupies the slot previously held by reservedForUpgrades2.

if (!roi.valid())
return true;
const math::XYZVector x(roi.vx(), roi.vy(), roi.vz());
if ((minRadius_ >= 0.0 && (x - bs).rho() < minRadius_) ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you really need the check of being a positive value ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right! The check has been removed.

vector<math::XYZVector> &decayVertices) const {
for (const auto &genParticle : genParticles) {
if (genMatch_ == "HToSSTobbbb") {
if (genParticle.pdgId() != 5)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't you need an abs ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is obviated by the removal of genMatch_, but in case you're curious, each decay vertex in this signal produces a b quark and b anti-quark, so we only need to look at the production point of the b quark (or the anti-quark, but not both). If abs were included, the vector of decay vertices would have duplicates.

continue;
if (!genParticle.isHardProcess())
continue;
if (abs(genParticle.mother()->pdgId()) != 9000006)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you mind to add a comment just above this in order to help other users to identify the particle ? (it seems like a not SM ones ...but I might be wrong, I do not play with the pdgID since long time)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviated by removal of genMatch_.

if (abs(genParticle.mother()->pdgId()) != 9000006)
continue;
} else if (genMatch_ == "stopToBottom") {
if (abs(genParticle.pdgId()) != 5)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here you have the abs, for instance ... as I was expecting

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this signal, each decay vertex produces a b quark or anti-quark and a charged lepton. So we have to look at the production points of both the quark and anti-quark in order to get all the decay vertices.

const math::XYZVector bs(beamSpot->x0(), beamSpot->y0(), beamSpot->z0());
event.getByToken(trackClustersToken_, trackClusters);
vector<math::XYZVector> decayVertices;
if (genMatch_ != "None") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slava77 @jpata is it ok to have in RECO code some dependency on gen ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, we do have some examples of dependence on MC truth objects in reco modules.
Still, it is not really a welcome dependence and if possible, it would be better to decouple.
clearly, the default configuration is not supposed to use MC-specific inputs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes sense to decouple the MC truth dependence, which was only used for testing anyway, so I've removed everything related to genMatch_.

}

const double ExoRegionProducer::getDiscriminatorValue(const ExoRegion &roi, const reco::BeamSpot &bs) const {
tensorflow::Tensor vertexTensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({1, 40, 23}));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you add a comment about these "magic numbers", please ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added some comments and also attempted to assign the magic numbers to meaningfully named constants.

tensorflow::Tensor vertexTensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({1, 40, 23}));
auto vertex_map = vertexTensor.tensor<float, 3>();
tensorflow::Tensor annulusTensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({1, 10, 8}));
auto annulus_map = annulusTensor.tensor<float, 3>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as above

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@slava77
Copy link
Contributor

slava77 commented Feb 3, 2021

the .pb model files will have to be (eventually) removed from this PR commit history.
we'd need to first sort out in which package to keep this code (if ExoRegionalTracking is needed, instead of some existing package)

tensorflow::Tensor annulusTensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({1, 10, 8}));
auto annulus_map = annulusTensor.tensor<float, 3>();

for (int i = 0, map_i = 0; map_i < 40; i++, map_i++) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wouldn't be better to have some constexpr for handling these "magic numbers" ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm familiar with constexpr, but I'm not sure how I could use it to handle the magic numbers in a clear way.

trackClustersToken_ =
consumes<edm::View<reco::VertexCompositeCandidate> >(cfg.getParameter<edm::InputTag>("trackClusters"));
if (genMatch_ != "None")
genParticlesToken_ = consumes<edm::View<reco::GenParticle> >(cfg.getParameter<edm::InputTag>("genParticles"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't you use the mayConsume ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't you use the mayConsume ?

No. The code below gets genParticlesToken_ under the same condition as the consumes declaration here, so consumes() is correct.

(and in practice mayConsume() currently does the same and consumes(), and to make it really functional could require interface changes, possibly similar to EventSetup mayConsume()).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviated by removing genMatch_.

@mtosi
Copy link
Contributor

mtosi commented Feb 3, 2021

the .pb model files will have to be (eventually) removed from this PR commit history.
we'd need to first sort out in which package to keep this code (if ExoRegionalTracking is needed, instead of some existing package)

I was going to write the same, indeed (thanks @slava77 )

@mtosi
Copy link
Contributor

mtosi commented Feb 3, 2021

one should understand if this PR can go like it is, so enabling this extra step by default
or instead if we want it as technical PR, perform a validation and then enabling

the quoted 7% increase seems reasonable, I do not know if one might think to have it run only on specific PDs (I hope not, but in case it could help)
@slava77 @jpata what do you think ?
the code per se seems already in a very good shape and I do not really see many places for improving the timing

as far as I understand @aehart already drop his tracks from the PF,
so these extra tracks will probably go into the lostTracks collection in the miniAOD,
because even if we do not have a highPurity flag for them I'm expecting their pT to be larger than the 2 GeV defined by a recent PR by the MUO POG. am I wrong ?

btw, I've just realized I forgot to say "thanks @aehart !"

@slava77
Copy link
Contributor

slava77 commented Feb 3, 2021

the quoted 7% increase seems reasonable, I do not know if one might think to have it run only on specific PDs (I hope not, but in case it could help)
@slava77 @jpata what do you think ?
the code per se seems already in a very good shape and I do not really see many places for improving the timing

I expect that ability to turn this on and off until after one or two relval cycles will be necessary.

7% is still a pretty significant setback in the CPU budget and some careful scrutiny is needed. Reduction in CPU in some other tracking parts like deep core can be considered as a balance.

@cmsbuild
Copy link
Contributor

cmsbuild commented Feb 3, 2021

-1

Failed Tests: UnitTests RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b0836f/12692/summary.html
COMMIT: be0cda6
CMSSW: CMSSW_11_3_X_2021-02-03-0800/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/32806/12692/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test unitTestsGroup_1 had ERRORS
---> test unitTestsGroup_4 had ERRORS
---> test TestFWCoreIntegrationStandalone had ERRORS
---> test testAlignmentOfflineValidation had ERRORS
and more ...

RelVals

  • 5.15.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log
  • 140.56140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log
  • 136.8523136.8523_RunJetHT2018C_nanoULremini+RunJetHT2018C_nanoULremini+NANOEDM2018_106Xv2+HARVESTNANOAOD2018_106Xv2/step2_RunJetHT2018C_nanoULremini+RunJetHT2018C_nanoULremini+NANOEDM2018_106Xv2+HARVESTNANOAOD2018_106Xv2.log
Expand to see more relval errors ...

RelVals-INPUT

  • 4.64.6_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1/step2_MinimumBias2010A+MinimumBias2010A+RECOSKIMALCA+HARVESTDR1.log
  • 136.72411136.72411_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM/step2_RunJetHT2016B_reminiaodUL+RunJetHT2016B_reminiaodUL+REMINIAOD_data2016UL_HIPM+HARVESTDR2_REMINIAOD_data2016UL_HIPM.log
  • 136.7611136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log
Expand to see more relval errors ...

AddOn Tests

  • fastsimcmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Wed Feb 3 22:05:17 2021-date Wed Feb 3 22:04:51 2021 s - exit: 256
  • hlt_data_Fake2cmsDriver.py RelVal -s HLT:Fake2,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run2_data_Fake2 --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run2_2016 --processName=HLTRECO --filein file:RelVal_Raw_Fake2_DATA.root --fileout file:RelVal_Raw_Fake2_DATA_HLT_RECO.root : FAILED - time: date Wed Feb 3 22:06:51 2021-date Wed Feb 3 22:04:55 2021 s - exit: 256
  • hlt_data_PRefcmsDriver.py RelVal -s HLT:PRef,RAW2DIGI,L1Reco,RECO --data --scenario=pp -n 10 --conditions auto:run3_data_PRef --relval 9000,50 --datatier "RAW-HLT-RECO" --eventcontent FEVTDEBUGHLT --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --era Run3 --processName=HLTRECO --filein file:RelVal_Raw_PRef_DATA.root --fileout file:RelVal_Raw_PRef_DATA_HLT_RECO.root : FAILED - time: date Wed Feb 3 22:11:15 2021-date Wed Feb 3 22:04:57 2021 s - exit: 256
Expand to see more addon errors ...

@slava77
Copy link
Contributor

slava77 commented May 20, 2021

@cmsbuild please test

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b0836f/15212/summary.html
COMMIT: 69445fe
CMSSW: CMSSW_12_0_X_2021-05-20-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32806/15212/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 3 differences found in the comparisons
  • DQMHistoTests: Total files compared: 37
  • DQMHistoTests: Total histograms compared: 2650486
  • DQMHistoTests: Total failures: 6
  • DQMHistoTests: Total nulls: 1
  • DQMHistoTests: Total successes: 2650457
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 1.798 KiB( 36 files compared)
  • DQMHistoSizes: changed ( 1000.0,... ): 0.078 KiB Tracking/MessageLog
  • DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
  • Checked 155 log files, 37 edm output root files, 37 DQM output files
  • TriggerResults: no differences found

@jfernan2
Copy link
Contributor

+1

@silviodonato
Copy link
Contributor

@aehart please note there is a conflict

@slava77
Copy link
Contributor

slava77 commented May 27, 2021

@aehart please note there is a conflict

in addition to this please also remove the .pb files from this PR commit history
(we can now use https://github.com/cms-data/RecoTracker-DisplacedRegionalTracking V00-01-00)

Also, to follow up on #32806 (comment), for better self-documentation please update the readme file https://github.com/cms-data/RecoTracker-DisplacedRegionalTracking/blob/main/README.md with some brief details on the content of the .pb files and how they were obtained

@slava77
Copy link
Contributor

slava77 commented Jun 10, 2021

@aehart
please clarify on your plans to update this PR.
Thank you.

@mtosi (in case you know)

@qliphy
Copy link
Contributor

qliphy commented Jun 29, 2021

kindly ping @aehart

@slava77
Copy link
Contributor

slava77 commented Jul 16, 2021

-1

unattended for more than two months now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants