Skip to content

Boosted DeepTauID v2.0 #46112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

mbluj
Copy link
Contributor

@mbluj mbluj commented Sep 24, 2024

PR description:

This PR adds DeepTauID dedicated to boostedTaus (aka taus in boosted final states).
This new tauID is run at miniAOD step and added to boostedTau collections at miniAOD and nanoAOD. It can be also run on-the-fly as part on nanoAOD step with existing miniAOD samples (for validation purposes) with a new era modifier run3_nanoAOD_pre142X introduced in #46066.

Expected changes:

  • three new tauIDs in the slimmedTausBoosted collection in miniAOD (with negligible impact on miniAOD size);
  • three new tauIDs (three new columns) in the boostedTau table in nanoAOD and relaxed boostedTau pre-selection (pt threshold 40->25GeV, boostedDeepTauID added to selection criteria OR'ed with previous tauIDs) - increase of size of the boostedTau table by ~70% and of size of nanoAOD by ~0.5% (tested with Run3 ttbar MC).

This PR depends on DeepTauID model files added in cms-data/RecoTauTag-TrainingFiles#14.

Most recent presentation on the BoostedDeepTauID: https://indico.cern.ch/event/1444695/#2-deepboostedtau-id-for-hh-bbt

PR validation:

Validated with wf. 13034.21 (TTbar_14TeV+2024PU_ProdLike)
and custom nanoAOD wf with 2022 miniAOD and run3_nanoAOD_pre142X modifier with cmsDriver command:

cmsDriver.py myNANO2022 -s NANO --mc --conditions auto:phase1_2022_realistic --era Run3,run3_nanoAOD_pre142X --eventcontent NANOAODSIM --datatier NANOAODSIM -n 1000 --no_exec --filein file:/eos/cms/store/group/phys_tau/kandroso/miniAOD/Run3_2022/TTtoLNu2Q.root

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport not planned

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 24, 2024

cms-bot internal usage

@mbluj
Copy link
Contributor Author

mbluj commented Sep 24, 2024

FYI @gparida @Ksavva1021 @pritampalit

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46112/41901

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @mbluj for master.

It involves the following packages:

  • Configuration/Eras (operations)
  • Configuration/StandardSequences (operations)
  • PhysicsTools/NanoAOD (xpog)
  • PhysicsTools/PatAlgos (xpog, reconstruction)
  • RecoTauTag/RecoTau (reconstruction)

@antoniovilela, @cmsbuild, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @jfernan2, @mandrenguyen, @rappoccio can you please review it and eventually sign? Thanks.
@AlexDeMoor, @AnnikaStein, @GiacomoSguazzoni, @JanFSchulte, @Martin-Grunewald, @Ming-Yan, @Senphy, @VinInn, @VourMa, @ahinzmann, @andrzejnovak, @azotz, @castaned, @demuller, @dgulhan, @fabiocos, @felicepantaleo, @gkasieczka, @gouskos, @gpetruc, @hatakeyamak, @jdamgov, @jdolen, @makortel, @mariadalfonso, @mbluj, @missirol, @mmarionncern, @mmusich, @mtosi, @nhanvtran, @rappoccio, @rovere, @sameasy, @schoef, @seemasharmafnal, @slomeo this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@jfernan2
Copy link
Contributor

enable nano

@jfernan2
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests RelVals RelVals-INPUT RelVals-NANO
Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-70ad07/41726/summary.html
COMMIT: 4484169
CMSSW: CMSSW_14_2_X_2024-09-24-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 7 errors in the following unit tests:

---> test testTauEmbeddingWorkflow2016postVFP had ERRORS
---> test testTauEmbeddingWorkflow2016preVFP had ERRORS
---> test testTauEmbeddingWorkflow2017 had ERRORS
and more ...

RelVals

----- Begin Fatal Exception 24-Sep-2024 17:27:01 CEST-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DeepTauId label='boostedDeepTau20161718v2p0BoostedForMini'
Exception Message:
edm::FileInPath unable to find file RecoTauTag/TrainingFiles/data/BoostedDeepTauId/boosteddeepTau_RunIIv2p0_core.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/CMSSW_14_2_X_2024-09-24-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/CMSSW_14_2_X_2024-09-24-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/CMSSW_14_2_X_2024-09-24-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_2_X_2024-09-24-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_2_X_2024-09-24-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_2_X_2024-09-24-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/4.53_RunPhoton2012B
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 24-Sep-2024 17:27:10 CEST-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DeepTauId label='boostedDeepTau20161718v2p0BoostedForMini'
Exception Message:
edm::FileInPath unable to find file RecoTauTag/TrainingFiles/data/BoostedDeepTauId/boosteddeepTau_RunIIv2p0_core.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/CMSSW_14_2_X_2024-09-24-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/CMSSW_14_2_X_2024-09-24-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/CMSSW_14_2_X_2024-09-24-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_2_X_2024-09-24-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_2_X_2024-09-24-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_2_X_2024-09-24-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/135.4_ZEEFS_13
----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 24-Sep-2024 17:27:58 CEST-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=DeepTauId label='boostedDeepTau20161718v2p0BoostedForMini'
Exception Message:
edm::FileInPath unable to find file RecoTauTag/TrainingFiles/data/BoostedDeepTauId/boosteddeepTau_RunIIv2p0_core.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/CMSSW_14_2_X_2024-09-24-1100/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/CMSSW_14_2_X_2024-09-24-1100/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46112/41726/CMSSW_14_2_X_2024-09-24-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_2_X_2024-09-24-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_2_X_2024-09-24-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_2_X_2024-09-24-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/140.023_RunZeroBias2022B
----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

RelVals-INPUT

  • 4.64.6_MinimumBias2010A/step2_MinimumBias2010A.log
  • 136.901136.901_RunDoubleMuon2016C/step2_RunDoubleMuon2016C.log
  • 136.902136.902_RunDoubleMuon2016H/step2_RunDoubleMuon2016H.log
Expand to see more relval errors ...

RelVals-NANO

  • 2500.0032500.003_NANOmc106Xul18v2/step2_NANOmc106Xul18v2.log
  • 2500.0022500.002_NANOmc106Xul17v2/step2_NANOmc106Xul17v2.log
  • 2500.0012500.001_NANOmc106Xul16v2/step2_NANOmc106Xul16v2.log
Expand to see more relval errors ...

@mbluj
Copy link
Contributor Author

mbluj commented Sep 24, 2024

It should be tested with cms-data/RecoTauTag-TrainingFiles#14 otherwise model files are not available.

@jfernan2
Copy link
Contributor

please test with cms-data/RecoTauTag-TrainingFiles#14

@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 8, 2024

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46112/42129

@cmsbuild
Copy link
Contributor

cmsbuild commented Oct 8, 2024

Pull request #46112 was updated. @cmsbuild can you please check and sign again.

@mbluj
Copy link
Contributor Author

mbluj commented Oct 8, 2024

Conflicts solved.

@mandrenguyen
Copy link
Contributor

@jfernan2 Please also sign cms-data/RecoTauTag-TrainingFiles#14

@mandrenguyen
Copy link
Contributor

please test

@smuzaffar
Copy link
Contributor

please test

@mbluj
Copy link
Contributor Author

mbluj commented Oct 11, 2024

Tests started 18h ago and not results yet, should the tests be restarted?

@hqucms
Copy link
Contributor

hqucms commented Oct 14, 2024

please abort

@hqucms
Copy link
Contributor

hqucms commented Oct 14, 2024

please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-70ad07/42171/summary.html
COMMIT: 8872b79
CMSSW: CMSSW_14_2_X_2024-10-14-1100/el8_amd64_gcc12
Additional Tests: PROFILING,NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46112/42171/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 20 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 106 differences found in the comparisons
  • DQMHistoTests: Total files compared: 44
  • DQMHistoTests: Total histograms compared: 3331066
  • DQMHistoTests: Total failures: 4151
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3326895
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 6.944000000000002 KiB( 43 files compared)
  • DQMHistoSizes: changed ( 11634.0,... ): 0.434 KiB Physics/NanoAODDQM
  • Checked 193 log files, 163 edm output root files, 44 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially added 356 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 537 differences found in the comparisons
  • DQMHistoTests: Total files compared: 21
  • DQMHistoTests: Total histograms compared: 55028
  • DQMHistoTests: Total failures: 305
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 54723
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 5.208000000000001 KiB( 20 files compared)
  • DQMHistoSizes: changed ( 2500.001,... ): 0.434 KiB Physics/NanoAODDQM
  • Checked 102 log files, 58 edm output root files, 21 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.001 2.960 2.947 0.013 ( +0.4% ) 2.92 3.10 -5.8% 6.719 6.055
2500.002 3.069 3.056 0.013 ( +0.4% ) 2.60 2.76 -5.5% 7.071 6.411
2500.003 3.011 2.998 0.013 ( +0.4% ) 2.72 2.89 -5.9% 7.100 6.487
2500.011 1.539 1.533 0.006 ( +0.4% ) 4.56 4.80 -5.1% 2.448 2.421
2500.012 2.045 2.030 0.015 ( +0.7% ) 2.76 2.90 -5.0% 2.643 2.605
2500.013 1.881 1.872 0.009 ( +0.5% ) 3.85 4.10 -6.0% 2.544 2.514
2500.021 0.022 0.022 0.000 ( +0.0% ) 0.89 0.98 -9.3% 2.409 2.381
2500.022 0.022 0.022 0.000 ( +0.0% ) 0.84 0.93 -9.6% 2.408 2.376
2500.023 0.022 0.022 0.000 ( +0.0% ) 0.83 0.93 -11.1% 2.275 2.244
2500.024 0.022 0.022 0.000 ( +0.0% ) 0.64 0.71 -10.3% 2.486 2.458
2500.031 0.035 0.035 0.000 ( +0.0% ) 0.77 0.86 -10.4% 2.479 2.451
2500.032 0.036 0.036 0.000 ( +0.0% ) 0.79 0.88 -10.4% 2.436 2.412
2500.033 0.037 0.037 0.000 ( +0.0% ) 0.70 0.80 -11.4% 2.526 2.492
2500.034 0.036 0.036 0.000 ( +0.0% ) 0.73 0.80 -9.9% 2.494 2.468
2500.101 2.746 2.728 0.018 ( +0.7% ) 7.18 9.00 -20.2% 7.036 6.339
2500.111 1.395 1.386 0.009 ( +0.7% ) 16.44 20.28 -19.0% 2.291 2.232
2500.112 1.815 1.803 0.011 ( +0.6% ) 13.22 15.29 -13.6% 2.366 2.313
2500.131 0.747 0.747 0.000 ( +0.0% ) 17.22 18.38 -6.3% 1.499 1.474
2500.201 2.573 2.552 0.022 ( +0.9% ) 6.19 7.58 -18.4% 6.317 5.610
2500.211 1.713 1.699 0.014 ( +0.8% ) 14.35 18.01 -20.3% 2.362 2.281
2500.212 2.108 2.095 0.013 ( +0.6% ) 11.79 14.29 -17.5% 2.438 2.361
2500.221 1.998 1.998 0.000 ( +0.0% ) 6.26 7.76 -19.4% 2.527 2.444
2500.222 3.300 3.276 0.024 ( +0.7% ) 6.22 7.62 -18.3% 2.614 2.527
2500.223 8.987 8.963 0.024 ( +0.3% ) 2.38 2.66 -10.5% 2.646 2.409
2500.224 5.834 5.810 0.024 ( +0.4% ) 0.57 0.60 -5.6% 2.678 2.130
2500.225 5.852 5.827 0.024 ( +0.4% ) 0.56 0.59 -4.2% 2.699 2.146
2500.226 3.062 3.038 0.024 ( +0.8% ) 6.18 7.60 -18.6% 2.617 2.411
2500.227 1.437 1.437 0.000 ( +0.0% ) 11.17 11.88 -6.0% 1.430 1.434
2500.231 1.403 1.403 0.000 ( +0.0% ) 11.81 13.54 -12.8% 2.262 2.180
2500.232 2.314 2.300 0.013 ( +0.6% ) 11.67 14.03 -16.8% 2.350 2.275
2500.233 4.760 4.747 0.013 ( +0.3% ) 3.93 4.26 -7.8% 2.375 2.279
2500.234 3.531 3.518 0.013 ( +0.4% ) 0.74 0.77 -3.9% 2.391 2.070
2500.235 3.543 3.530 0.013 ( +0.4% ) 0.72 0.75 -3.8% 2.406 2.088
2500.236 2.158 2.145 0.013 ( +0.6% ) 11.89 13.72 -13.3% 2.352 2.268
2500.237 1.016 1.016 0.000 ( +0.0% ) 16.35 17.05 -4.1% 1.469 1.459
2500.241 9.404 9.404 0.000 ( +0.0% ) 3.42 3.74 -8.6% 1.921 1.951
2500.242 10.331 10.331 0.000 ( +0.0% ) 0.83 0.91 -8.5% 1.709 1.735
2500.243 2.712 2.712 0.000 ( +0.0% ) 8.07 8.39 -3.7% 1.079 1.077
2500.244 485.976 485.976 0.000 ( +0.0% ) 0.53 0.56 -5.6% 1.674 1.692
2500.245 823.202 823.202 0.000 ( +0.0% ) 0.70 0.75 -5.5% 1.662 1.664
2500.901 1.777 1.777 0.000 ( +0.0% ) 19.87 21.53 -7.7% 1.842 1.846
2500.902 1.626 1.626 0.000 ( +0.0% ) 20.88 21.05 -0.8% 1.771 1.773
2500.911 13.995 13.995 0.000 ( +0.0% ) 2.12 2.92 -27.5% 1.100 1.095
2500.912 0.310 0.199 0.111 ( +55.6% ) 1.39 1.77 -21.7% 0.976 0.974
2500.913 0.110 0.110 0.000 ( +0.0% ) 1.14 1.17 -2.5% 0.980 0.972

@mandrenguyen
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit ceae18e into cms-sw:master Oct 15, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants