Add JES up/down systematic evaluation

[watson-ij]

No description provided.

[watson-ij]

nanoAOD-tools seems to have a python version of this

https://github.com/cms-nanoAOD/nanoAOD-tools/tree/master/python/postprocessing/modules/jme

[jshlee]

since @quark2 needs this the most, can you check this

[quark2]

If you run the following after full recipe for production, then (it will take about 20 minutes, depending on the number of events, of course) you can obtain nanoAOD_[...]_Skim.root;

cd $CMSSW_BASE/src/PhysicsTools/NanoAODTools
python scripts/nano_postproc.py ./ /xrootd/store/group/nanoAOD/run2_2016v5/.../.../.../.../nanoAOD_[...].root --import=PhysicsTools.NanoAODTools.postprocessing.modules.jme.jetmetUncertainties jetmetUncertainties2016All

The output file nanoAOD_[...]_Skim.root contains all of the source, including new branches of shifted jet and MET like the followings:

Jet_pt_nom
Jet_corr_JEC
Jet_corr_JER
Jet_mass_nom
MET_pt_nom
MET_phi_nom
Jet_pt_jerUp
Jet_mass_jerUp
Jet_mass_jmrUp
Jet_mass_jmsUp
MET_pt_jerUp
MET_phi_jerUp
Jet_pt_jerDown
Jet_mass_jerDown
Jet_mass_jmrDown
Jet_mass_jmsDown
MET_pt_jerDown
MET_phi_jerDown
Jet_pt_jesTotalUp
Jet_mass_jesTotalUp
MET_pt_jesTotalUp
MET_phi_jesTotalUp
Jet_pt_jesTotalDown
Jet_mass_jesTotalDown
MET_pt_jesTotalDown
MET_phi_jesTotalDown

and so on.

I think this is what we are looking for. See https://github.com/cms-nanoAOD/nanoAOD-tools/blob/master/python/postprocessing/modules/jme/jetmetUncertainties.py#L159, where the analyse() is the main part of the works. Also we need to check what is shaken for each of these evaluations.

If everything so far is okay, then we need to consider other problems: how to apply. For run2_2016v5, we can overwrite the existing root files by the one with the systematic evaluations. Or, we can make 'friend tree files' (This nanoAOD post-process program supports this) and then keep them separately. Also we need to think about how to apply this to new productions, such as run2_2016v6 or run2_2017vX (how to run this in crab job?)

[quark2]

One problem of the above solution is that the 'python' code is seriously slow. I estimated the requested time and obtained that it will take about 30 hours (NOT 30 minutes!) with 700 cores! I think it's because the post-process code consists of 'pure python' code, i.e., it runs the loop over events and calculates in python, not in a built-in functions. Of course, we have crab, so it might not be a problem, but to me it's not satisfied. Maybe porting the code into C/C++ or using a 'compiler' for python (there are such things, you know) can be solutions for this situ.

Meanwhile, for the next versions of nanoAOD we can find a neat solution which lives in CATTools; see https://github.com/vallot/CATTools/blob/cat80x/CatProducer/plugins/CATJetProducer.cc#L219-L226 and https://github.com/vallot/CATTools/blob/cat80x/CatProducer/plugins/CATJetProducer.cc#L236-L265. If we apply these prescriptions to our producer now, we will have the systematic evaluations in the next versions without any post stuffs

[quark2]

See a new pull request: #114; I added the codes for jet uncertainty evaluator used in CATTools

[quark2]

Alright, most of most of my code (and code in CATTools) agrees with the stuffs in nanoAOD, except for the additional mass scale factor. See https://github.com/cms-nanoAOD/nanoAOD-tools/blob/master/python/postprocessing/modules/jme/jetSmearer.py#L176, which evaluates jet_mass_jmrUpVal, while the nominal scale factor is 0.1, not 1.0 (see the line 162 in the code)! According to the evaluator I think 0.1 makes no sense, because the nominal value has to be closer to the reco jet as much as possible, right? Also I cannot find any description in twiki pages about the additional jet scale factor (not only jet_mass_jmrNomVal, but also jet_mass_jmr[Up, Down]Val). What is the basis for this variable?

(FYI, nanoAOD guys refered to the followings in their code: https://twiki.cern.ch/twiki/bin/view/CMS/JetResolution and https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookJetEnergyResolution for JER, and https://twiki.cern.ch/twiki/bin/view/CMS/JECDataMC and https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookJetEnergyCorrections#JetCorUncertainties for JES.)

[quark2]

I made codes for evaluation of JER/JES uncertainties on the fly; see 4aeabb7. I checked that my code reproduces the evaluations provided by nanoAOD guys (the postprocess tool in NanoAODTools). My code takes only few additional costs, so I think we can evaluate on the fly without any problem.
But still there are unsolved questions.

• What is the JER_nom factor? (See here and here.) Why are they not unit? Or should we apply this factor to our nominal jet pT...?
• Why did nanoAOD guys not use evaluation on JES along opposite direction? See this usage of nanoAOD guys, while CATTools guys are using evaluations along opposite direction as like this.
• Still dunno about additional JER nominal factors (jet_mass_jmrNomVal) on mass... (See here) I cannot find any docs about that. Can we ignore this? If the getSmearValsM() code of nanoAOD guys has a bug as I mentioned above, we may be able to ignore.

[watson-ij]

Did you look at the twikis on L238? What do they say?
L237: # evaluate JER scale factors and uncertainties
L238: # (cf. https://twiki.cern.ch/twiki/bin/view/CMS/JetResolution and https://twiki.cern.ch/twiki/bin/view/CMSPublic/WorkBookJetEnergyResolution )

[quark2]

First of all, I cannot find any of jet mass uncertainty, at least in CMS twiki. But also no mention about that in the TopJME. I think then we can ignore about jet mass uncertainty.
On the other hand, according to https://twiki.cern.ch/twiki/bin/view/CMS/JetResolution, in CMSSW there is a JER evaluation code for miniAOD (See here). But the way is somewhat different from that of nanoAOD tools. I found two things: First, the twiki page and CMSSW code are using additional condition (deltaR < R_cone/2, |pT - pT_gen| < 3 * s_JER * pT) for matching jet and genjet, while nanoAOD guys using just simple deltaR method (see here, where the usage of this can be found in here). Second, for jets without matched genjet nanoAOD guys are using other method than stochastic method (see here, I have no idea where it comes from...) So, I think it is better to abandon the method of nanoAOD guys and take the way in the twiki page which is used in CMSSW. Note that CATTools is using totally same way to the twiki page.
Btw, CATTools has one difference: They are using 'opposite direction' on JES (see here), as mentioned. But in CMSSW there is no usage of the opposite direction (i.e., no one uses getUncertainty(false).) I think replacing getUncertainty(false) by getUncertainty(true) might be okay.

[quark2]

One thing more: Note that in general the nominal value of s_JER (in JetResolution page) is not 1. According to TopJME twiki page seems like we need to apply the 'nominal factor' of JER to our 'nominal' (i.e., default) mode. Actually, many of analyser codes in CATTools uses jet kinematic variables with the nominal factor. I think we need to discuss and investigate about this.

[quark2]

I investigated whether nominal JER smearing is applied to our nanoAOD or not. According to the jet_cff.py, there is no nominal smearing. Only (L2)L3 JEC are applied (see here and here, of which the cfi codes can be found in here and here and the plugin codes are here and here; only JEC is found). But other analysis' are using nominal smeared jets as a nominal jets. I think we need to replace our jets by smeared ones.
Btw, I have no idea how to verify this. That is, I want to check that it really has no smearing but I don't know how to do. Is there someone who has an idea?