Renumber GPU workflows #259
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Reference before this PR: Validation summaryReference release CMSSW_10_4_0 at b8365c6
|
Validation summaryReference release CMSSW_10_4_0 at b8365c6
|
TTbar reference summary
TTbar #259 summary
|
fwyzard
pushed a commit
that referenced
this pull request
Oct 19, 2020
fwyzard
pushed a commit
that referenced
this pull request
Oct 23, 2020
fwyzard
pushed a commit
that referenced
this pull request
Nov 6, 2020
fwyzard
pushed a commit
that referenced
this pull request
Nov 16, 2020
fwyzard
added a commit
that referenced
this pull request
Nov 27, 2020
fwyzard
added a commit
that referenced
this pull request
Dec 26, 2020
fwyzard
pushed a commit
that referenced
this pull request
Dec 29, 2020
silviodonato
pushed a commit
to cms-sw/cmssw
that referenced
this pull request
Apr 12, 2021
* Add workflows for Riemann fit and GPU (cms-patatrack#20) * add `riemannFit` and `gpu` modifiers and workflows for Riemann fit and GPU modules * switch GPU modules with `gpu` modifier * Add a DQM sequence for pixel-only tracking (cms-patatrack#23) * add a DQM sequence for pixel-only tracking * add pixelTrackingOnlyDQM sequences to the pixelTrackingOnly workflows * add pixelTrackingOnlyDQM sequences to the riemannFit and gpu workflows * Synchronise with CMSSW_10_1_0_pre2 * Synchronise with CMSSW_10_1_0_pre3 * Synchronise with CMSSW_10_1_0 * Synchronise with CMSSW_10_2_0_pre1 * Synchronise with CMSSW_10_2_0_pre2 * Synchronise with CMSSW_10_2_0_pre3 * Synchronise with CMSSW_10_2_0_pre4 * Synchronise with CMSSW_10_2_0_pre5 * Port the Riemann fit to CUDA (cms-patatrack#60) - the CPU Riemann fit works using all combinations between the 2 booleans: `useErrors` and `useMultipleScattering`; - the standalone version of the GPU Riemann fit has been updated in order to explore all possibilities among the 2 booleans above: all of them work and produce identical results up to 1e-5 precision (the default one, 1e-6 fails when enabling multiScattering, most likely due to matrix inversions); - the GPU version of the Riemann fit within CMSSW works, with 1 fit assigned to each thread, with 32 threads/warps, all dynamically computed. Things that needs a "hack": - limit the "dynamic" size of Eigen matrices to at most, 4x4, which is just fine for quadruplets. Using anything wider will cause errors which I *believe* is related to the stack size of threads on the GPU; - cast matrices to be inverted to 4x4 (was done before the previous point: will revert it back and see if that's still needed or not, but I believe it is); this was done in order to "specialize" the `invert()` call to something that is "natively" supported by Eigen on GPU (that brought in also few `__host__` `__device__` here and there in Eigen); - fix the alignment of the `struct` holding the results of the fit, since its size was different on GPU and CPU, causing an annoying off-by-one effect. * Synchronise with CMSSW_10_2_0_pre6 * update relval input strings (102X) * Synchronise with CMSSW_10_2_0 * adding NANO step to 2018 MC workflows * Merge pull request #24024 from prebello/from-CMSSW_10_2_X_2018-07-24-2300 update relval input strings (102X) * Synchronise with CMSSW_10_2_2_patch1 * activate the new SiStripCalSiStripCalBiasScan in the matrix test * Add pixel tracking workflows for data (cms-patatrack#144) * add HEM-15-16 relval workflows * introduce RelVal for hybrid emulation (with ZS10 repacking),HLT-like zero-suppression (with normal ZS repacking),RECO of 2015 PbPb VirginRaw data * add 3 production workflows for premix2018 * Merge pull request #24550 from zhenhu/add-HEM-15-16-relvals_102X [10.2.X] add HEM-15-16 relval workflows * Merge pull request #24618 from zhenhu/addPremixProdWfs_102X backport [10.2.X] add 3 production workflows for premix2018 * Synchronise with CMSSW_10_2_5 * Synchronise with CMSSW_10_2_6 * Synchronise with CMSSW_10_4_0_pre2 * Synchronise with CMSSW_10_4_0_pre3 * Synchronise with CMSSW_10_4_0_pre4 * Synchronise with CMSSW_10_4_0 * Change GPU workflow numbering: .7->.51, .8->.52, .9->.53 (cms-patatrack#259) * Synchronise with CMSSW_10_5_0_pre1 * Synchronise with CMSSW_10_5_0_pre2 * Synchronise with CMSSW_10_6_0_pre2 * Rework the Riemann fit and broken line fit (cms-patatrack#338) Merge the Riemann and broken line fits into single configurable pixel n-tuplet fitter, and extend it to work with up to 5 hits. Mmake the broken line fit the default algorithm. Try both triplets and quadruplets in the pixel "hole". Limit pT used to compute the multple scattering. Use the inline Cholesky decomposition. Generic clean up and improvements. * Synchronise with CMSSW_10_6_0 * Synchronise with CMSSW_11_0_0_pre7 * Fix broken merge in Configuration/PyReleaseValidation * Synchronise with CMSSW_11_0_0_pre11 * Update and extend the Patatrack workflows Update the definition of the Patatrack workflows to avoid conflicts in CMSSW_11_0_0_pre12: - renumber to avoid conflicts with #27392 - reimplement the definition following #28281 - add Z->mumu 2018 realistic workflows - add TTbar and Z->mumu 2021 realistic workflows - move the data workflows to 2018D, HLTPhysics and JetHT * Synchronise with CMSSW_11_0_0_pre13 * Backport: add ECAL-only workflows for 2018 data (#30105) * Add tracking-only and pixel tracking-only workflows for Run 3 (#30338) * Backport: add ECAL-only and HCAL-only workflows for MC and data (#30350) Backport #30105: add ECAL-only workflows for data. Backport #30136: add HCAL-only workflows for MC and data. * Synchronise with CMSSW_11_2_0_pre2 * Update the RelVal workflows and the CPU customisation (cms-patatrack#549) Update the RelVal workflows and the CPU customisation: - change the .501 workflow to run the full Patatrack pixel track reconstruction on CPU - add a customisation to run the Patatrack reconstruction with triplets, on CPU and GPU - add the .505 and .506 workflows to reconstruct triplets, on CPU and GPU Co-authored-by: Andrea Bocci <[email protected]> * Synchronise with CMSSW_11_2_0_pre8 * Synchronise with CMSSW_11_2_0_pre9 * Synchronise with CMSSW_11_2_0_pre10 * Backport: add RecoFakeHLT steps to 2018 Patatrack WFs to enable GPU modifiers (#32333) The 2018 GPU matrix WFs (10824.502, 10824.512, 10824.522, 10842.502) produced the same cmsDriver configurations as their CPU counterparts. This PR fixes this by setting the gpu modifier also for the RecoFakeHLT step. * Synchronise with CMSSW_11_3_0_pre1 * Synchronise with CMSSW_11_3_X_2020-12-24-2300 * Document the new Pixel workflows Co-authored-by: Matti Kortelainen <[email protected]> Co-authored-by: Marco Rovere <[email protected]> Co-authored-by: Patricia <[email protected]> Co-authored-by: Francesco <[email protected]> Co-authored-by: cmsbuild <[email protected]> Co-authored-by: mmusich <[email protected]> Co-authored-by: zhen <[email protected]> Co-authored-by: Vincenzo Innocente <[email protected]> Co-authored-by: Mariarosaria D'Alfonso <[email protected]> Co-authored-by: AdrianoDee <[email protected]> Co-authored-by: Thomas Reis <[email protected]>
marco-link
added a commit
to marco-link/cmssw
that referenced
this pull request
Apr 21, 2021
* Code check * Add PPS geometry and simulation tags to GTs * Changing the proton propagator to the Optical Function method * Revert back to standard naming for Phase1 classes * Fixed include guards * Adding the new filter modules * Code format * Remove unused debug for-loop. * Adding the multiple daughters of same kind possibility * Use esConsumes for geometry and ETTmap records. * Codecheck update * Fix the deug statements * use old hadronic physics * Correcting the clang build err - corrected splelling in header guard * fixed compilation * add 14 TeV VBF H>inv pythia cfg * added update on low-energy GFlash * fix issues with clang tidy and static analyzer * First skeleton of a Trackster from Simcluster producer * Fix bug * Add simtracksters into the event content * Make SimTracksters part of TICL Tasks * Create a SimTrackster per SimCluster * Deleted methods should be public * keep multiclusters from simtracksters * cleanup * add multiclusters from simtrackstrs * add simtracksters to validation * Make multiplicity take the simulated energy fraction * add simTracksters iteration to validation * replacing tabs with spaces and speeding up --help * make enabling simclusters validation plots * cleanup * Add SimTracksters to FEVT and PreValidation Sequence * Fix for premix wf * get ES data through token * update and reorder BuildFile * adding missing dependencies in BuildFile * Patatrack integration - Pixel workflows (12/N) (cms-sw#31854) * Add workflows for Riemann fit and GPU (cms-patatrack#20) * add `riemannFit` and `gpu` modifiers and workflows for Riemann fit and GPU modules * switch GPU modules with `gpu` modifier * Add a DQM sequence for pixel-only tracking (cms-patatrack#23) * add a DQM sequence for pixel-only tracking * add pixelTrackingOnlyDQM sequences to the pixelTrackingOnly workflows * add pixelTrackingOnlyDQM sequences to the riemannFit and gpu workflows * Synchronise with CMSSW_10_1_0_pre2 * Synchronise with CMSSW_10_1_0_pre3 * Synchronise with CMSSW_10_1_0 * Synchronise with CMSSW_10_2_0_pre1 * Synchronise with CMSSW_10_2_0_pre2 * Synchronise with CMSSW_10_2_0_pre3 * Synchronise with CMSSW_10_2_0_pre4 * Synchronise with CMSSW_10_2_0_pre5 * Port the Riemann fit to CUDA (cms-patatrack#60) - the CPU Riemann fit works using all combinations between the 2 booleans: `useErrors` and `useMultipleScattering`; - the standalone version of the GPU Riemann fit has been updated in order to explore all possibilities among the 2 booleans above: all of them work and produce identical results up to 1e-5 precision (the default one, 1e-6 fails when enabling multiScattering, most likely due to matrix inversions); - the GPU version of the Riemann fit within CMSSW works, with 1 fit assigned to each thread, with 32 threads/warps, all dynamically computed. Things that needs a "hack": - limit the "dynamic" size of Eigen matrices to at most, 4x4, which is just fine for quadruplets. Using anything wider will cause errors which I *believe* is related to the stack size of threads on the GPU; - cast matrices to be inverted to 4x4 (was done before the previous point: will revert it back and see if that's still needed or not, but I believe it is); this was done in order to "specialize" the `invert()` call to something that is "natively" supported by Eigen on GPU (that brought in also few `__host__` `__device__` here and there in Eigen); - fix the alignment of the `struct` holding the results of the fit, since its size was different on GPU and CPU, causing an annoying off-by-one effect. * Synchronise with CMSSW_10_2_0_pre6 * update relval input strings (102X) * Synchronise with CMSSW_10_2_0 * adding NANO step to 2018 MC workflows * Merge pull request cms-sw#24024 from prebello/from-CMSSW_10_2_X_2018-07-24-2300 update relval input strings (102X) * Synchronise with CMSSW_10_2_2_patch1 * activate the new SiStripCalSiStripCalBiasScan in the matrix test * Add pixel tracking workflows for data (cms-patatrack#144) * add HEM-15-16 relval workflows * introduce RelVal for hybrid emulation (with ZS10 repacking),HLT-like zero-suppression (with normal ZS repacking),RECO of 2015 PbPb VirginRaw data * add 3 production workflows for premix2018 * Merge pull request cms-sw#24550 from zhenhu/add-HEM-15-16-relvals_102X [10.2.X] add HEM-15-16 relval workflows * Merge pull request cms-sw#24618 from zhenhu/addPremixProdWfs_102X backport [10.2.X] add 3 production workflows for premix2018 * Synchronise with CMSSW_10_2_5 * Synchronise with CMSSW_10_2_6 * Synchronise with CMSSW_10_4_0_pre2 * Synchronise with CMSSW_10_4_0_pre3 * Synchronise with CMSSW_10_4_0_pre4 * Synchronise with CMSSW_10_4_0 * Change GPU workflow numbering: .7->.51, .8->.52, .9->.53 (cms-patatrack#259) * Synchronise with CMSSW_10_5_0_pre1 * Synchronise with CMSSW_10_5_0_pre2 * Synchronise with CMSSW_10_6_0_pre2 * Rework the Riemann fit and broken line fit (cms-patatrack#338) Merge the Riemann and broken line fits into single configurable pixel n-tuplet fitter, and extend it to work with up to 5 hits. Mmake the broken line fit the default algorithm. Try both triplets and quadruplets in the pixel "hole". Limit pT used to compute the multple scattering. Use the inline Cholesky decomposition. Generic clean up and improvements. * Synchronise with CMSSW_10_6_0 * Synchronise with CMSSW_11_0_0_pre7 * Fix broken merge in Configuration/PyReleaseValidation * Synchronise with CMSSW_11_0_0_pre11 * Update and extend the Patatrack workflows Update the definition of the Patatrack workflows to avoid conflicts in CMSSW_11_0_0_pre12: - renumber to avoid conflicts with cms-sw#27392 - reimplement the definition following cms-sw#28281 - add Z->mumu 2018 realistic workflows - add TTbar and Z->mumu 2021 realistic workflows - move the data workflows to 2018D, HLTPhysics and JetHT * Synchronise with CMSSW_11_0_0_pre13 * Backport: add ECAL-only workflows for 2018 data (cms-sw#30105) * Add tracking-only and pixel tracking-only workflows for Run 3 (cms-sw#30338) * Backport: add ECAL-only and HCAL-only workflows for MC and data (cms-sw#30350) Backport cms-sw#30105: add ECAL-only workflows for data. Backport cms-sw#30136: add HCAL-only workflows for MC and data. * Synchronise with CMSSW_11_2_0_pre2 * Update the RelVal workflows and the CPU customisation (cms-patatrack#549) Update the RelVal workflows and the CPU customisation: - change the .501 workflow to run the full Patatrack pixel track reconstruction on CPU - add a customisation to run the Patatrack reconstruction with triplets, on CPU and GPU - add the .505 and .506 workflows to reconstruct triplets, on CPU and GPU Co-authored-by: Andrea Bocci <[email protected]> * Synchronise with CMSSW_11_2_0_pre8 * Synchronise with CMSSW_11_2_0_pre9 * Synchronise with CMSSW_11_2_0_pre10 * Backport: add RecoFakeHLT steps to 2018 Patatrack WFs to enable GPU modifiers (cms-sw#32333) The 2018 GPU matrix WFs (10824.502, 10824.512, 10824.522, 10842.502) produced the same cmsDriver configurations as their CPU counterparts. This PR fixes this by setting the gpu modifier also for the RecoFakeHLT step. * Synchronise with CMSSW_11_3_0_pre1 * Synchronise with CMSSW_11_3_X_2020-12-24-2300 * Document the new Pixel workflows Co-authored-by: Matti Kortelainen <[email protected]> Co-authored-by: Marco Rovere <[email protected]> Co-authored-by: Patricia <[email protected]> Co-authored-by: Francesco <[email protected]> Co-authored-by: cmsbuild <[email protected]> Co-authored-by: mmusich <[email protected]> Co-authored-by: zhen <[email protected]> Co-authored-by: Vincenzo Innocente <[email protected]> Co-authored-by: Mariarosaria D'Alfonso <[email protected]> Co-authored-by: AdrianoDee <[email protected]> Co-authored-by: Thomas Reis <[email protected]> * Remove CLCT BX and LCT BX comparisons * add valueMap producer for the track distances * add monitoring of closest track * use CTPPSRun2Geometry customization to fix HLT test with Run2 data * use the iEvent.getHandle syntax and make TrackDistanceValueMapProducer a global EDProducer * Use 112X geometry in online data global tags * Use 112X geometry in offline data global tags * fix in CTPPSRun2Geometry cust funt * fix in CTPPSRun2Geometry cust funt * fix HLT addOnTest to use CTPPSRun2Geometry custom funct * Disable the deug possibility * add 14 tev gluino model * Remove trackingMaterialGroups_ForPhaseII.xml from T25/T26 dict * add comment to deprecate Multiclusters * add name and date * Modernize code and simplify * add the possibility to save only the first nth closest track distances * fix container size * Add a new workflow with V15 version of HGCal Geometry * Modify Run3 workflows with changed version of RP devices * add spaces * Change nodigis to hasdigis * Hit counter for shower trigger * Use constants from CSCConstants * Use constants from CSCConstants * Added ZpEE and ZpMM 14TeV fragments * Use auto * Migrate EcalTPCondAnalyzer to use esConsumes and replace couts with LogVerbatim. * Move CSCConstants.h to DataFormats/CSCDigi * Better document the wiregroups * Delete particleFlowBadHcalPseudoCluster_cff.py Was added by accident instead of RecoParticleFlow/PFClusterProducer/python/particleFlowBadHcalPseudoCluster_cff.py * migrate DQM/TrackingMonitor to esConsumes * Move plugin out of SimDataFormats * Fix the memory error * Code format * First skeletton of the harvester * Added skeletton for the worker * Starting to populate a few monitors * Making use of detid's naming export features * Updated to the new PPS DataFormat accessors * Stripping the histogram containers from worker * Using rechits instead of digis * Filling the condition on pixel tracks * Full implementation of pixels tracks selection stage * Corrected RP station and pixel tracks cuts retrieval * Starting to integrate geometry * Full usage of db-loaded geometry for channels booking, starting to fill the harvester * Dropped CLHEP dependency * Also storing uncalibrated ToT distribution, first attempt to fit the t-vs-tot profile in harvester * Applied code-format * Inserted PPSTimingCalib selection/paths/output sequences in their right place * Delete ME file after copy * Small cleanup, filtered stream renaming * Reverted back filtered stream name * Matched new folders hierarchy * Updated paths * Added Geometry/Records to TimingCalibration build * Added an uncalibrated/calibrated collection of diamond information * PPS track filter stripped from PCL worker * Added a test sequence wrapping up all PCL operations * Cleanup * Per-pot track filter for PPS * More C++-friendly tracks counting * Replaced the track filter by the HLT-defined one * Cleanup * Cleanup of timing calibration worker sequence * Keeping the diamond digis instead of rechits to perform uncalibrated rechits "reconstruction" * Cleanup of harvester includes * Using a smart pointer to hold x-profile used in calibration fit * Ensure PoolDBService is available before processing * Delegating DB service availability check at plugin constructor level * Avoiding a couple of try/catch blocks through CTPPSDiamondDetId::check * Filters rules retrieved by reference instead of copies * Thread-safe geometry retrieval from ES * Code-format * Dropping the DB service private attribute to create it at constructor and for each end job * Transition type set for geometry consumes declaration * Thread-safe ES geometry condition retrieval for PCL harvester * Update CalibPPS/TimingCalibration/python/ALCARECOPPSCalTrackBasedSel_Output_cff.py Co-authored-by: Silvio Donato <[email protected]> * Update CalibPPS/TimingCalibration/python/ALCARECOPPSCalTrackBasedSel_cff.py Co-authored-by: Silvio Donato <[email protected]> * Cloning instead of deep-copying, Task/Sequence redefinition of the filtering/worker modules * Replaced back ','s with '*'s for seqALCARECOPPSCalTrackBasedSel Sequence * Using a reco task instead of a sequence for the taskALCARECOPPSTimingCalib task definition * PCL WF test is now 1041 * Apply changes from : scram build code-checks / code-format * use references * Correct validation code for MB of HCAL * Code check * Utilize a phase2 scenario for testing dd4hep scenarios * Clarify number of strips and half-strips * Tighten assert statements * put distrip and cfeb calculation in functions * Fix typo * Check in more detail the content of teh geometry * Code check * Clean BuildFiles * rename and small fixes * CLANG test * cleanup of Track analyzer moving double loop to dqm harvester * remove commented lines * remove unwanted includes and cout usage * remove duplicate includes and cout * resolve code check issues * Move the macros to a separate directory and adopt for dd4hep * Tryto fix * change logic in folded occupancy client to allow running for cosmics;update cosmics config * Code check * update HI sequence * apply coding style * reorganization in configs * Add step3 and 4 for TICL only * Move check for pixel active area into RectangularMTDTopology, make it usable for BTL as well * Fix post merge compilation errors * Code format * Fix indentation in EDAlias python dump * Add a new scenario D84 same as D70 but with updated tracker T24 * updated physics to 10.7 * removed obsolete * removed remaining obsolete * complete update * complete update * clean-up custom physics * code-checks * code-checks * update input file in fwliteWithPythonConfig_cfg.py * Remove dead essignments from L1Trigger/CSCTrackFinder * Suggestions by Andrea Perrotta * Suggestions by Andrea Perrotta * Modified version of EB numbering scheme code which will work with modified EB algorithm * deal with fastsim issue raised in cms-sw#33430 * Code check * Update python tests - py3 only * Code format * use python3 for all tests * Minor changes in the configs removing uneeded loads in view of the new GT * uncommented the verbosity line and changed the names * fixed the names * correct for gen fragment naming conventions * remove old cfgs Co-authored-by: Sunanda <[email protected]> Co-authored-by: Helena <[email protected]> Co-authored-by: cmsbuild <[email protected]> Co-authored-by: Luiz Mundim <[email protected]> Co-authored-by: Davide Valsecchi <[email protected]> Co-authored-by: Aravind <[email protected]> Co-authored-by: Thomas Reis <[email protected]> Co-authored-by: Vladimir <[email protected]> Co-authored-by: Vincenzo Innocente <[email protected]> Co-authored-by: Silvio <[email protected]> Co-authored-by: Theodor Christian Herwig <[email protected]> Co-authored-by: mmusich <[email protected]> Co-authored-by: Felice <[email protected]> Co-authored-by: Andrea Bocci <[email protected]> Co-authored-by: Matti Kortelainen <[email protected]> Co-authored-by: Marco Rovere <[email protected]> Co-authored-by: Patricia <[email protected]> Co-authored-by: Francesco <[email protected]> Co-authored-by: zhen <[email protected]> Co-authored-by: Mariarosaria D'Alfonso <[email protected]> Co-authored-by: AdrianoDee <[email protected]> Co-authored-by: Sven Dildick <[email protected]> Co-authored-by: Chris West <[email protected]> Co-authored-by: Adinda de Wit <[email protected]> Co-authored-by: Andrea <[email protected]> Co-authored-by: namppl <[email protected]> Co-authored-by: laurenhay <[email protected]> Co-authored-by: Leonardo Cristella <[email protected]> Co-authored-by: Laurent Forthomme <[email protected]> Co-authored-by: Laurent Forthomme <[email protected]> Co-authored-by: Carlo Battilana <[email protected]> Co-authored-by: Jonas Rembser <[email protected]> Co-authored-by: Suvankar Roy Chowdhury <[email protected]> Co-authored-by: Erica Brondolin <[email protected]> Co-authored-by: Fabio Cossutti <[email protected]> Co-authored-by: Mircho Rodozov <[email protected]> Co-authored-by: Mircho Nikolaev Rodozov <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
To fix a conflict in workflow numbers wrt. CMSSW
mastercms-sw#25647 (comment).I chose the following convention
based on all workflows being variants of the pixel-tracking-only (
.5).@fwyzard