Skip to content

MiniAOD from HLT Scouting #50307

Open
drkovalskyi wants to merge 33 commits intocms-sw:masterfrom
drkovalskyi:scout_miniaod
Open

MiniAOD from HLT Scouting #50307
drkovalskyi wants to merge 33 commits intocms-sw:masterfrom
drkovalskyi:scout_miniaod

Conversation

@drkovalskyi
Copy link
Copy Markdown
Contributor

PR description:

Convert Run3 scouting data to standard MiniAOD format so that analysts can use the same accessors regardless of whether objects came from standard reconstruction or scouting.

What this PR does:

  • Adds scouting-to-MiniAOD producers in PhysicsTools/PatFromScouting that convert Run3Scouting* objects to standard pat:: objects (slimmedMuons, slimmedElectrons, slimmedPhotons, slimmedJets, packedPFCandidates, etc.)
  • Implements scouting-to-reco conversion directly in pat:: object constructors, keeping the producers thin and the conversion logic co-located with the data format
  • Adds universal accessor pattern to pat::Electron, pat::Photon, and pat::Muon with isScouting*() flags that dispatch to scouting-specific or standard data internally
  • Embeds track details in PackedCandidates by matching scouting PF candidates to scouting tracks
  • Populates reco::Track hit patterns from scouting hit counts using a simple algorithm, which achieves 90% correct prediction of layer information
  • Resolves scouting jet constituent indices to CandidatePtr daughters so numberOfDaughters(), daughter(i), daughterPtr(i) work transparently
  • Provides full cmsDriver workflow: PAT:@scout (HLTSCOUT → MiniAOD) and NANO:@ScoutMini (MiniAOD → NanoAOD)

For details on motivation and content optimization see https://indico.cern.ch/event/1646385/#4-proposal-for-miniaod-from-hl

PR validation:

  • scram b compiles cleanly
  • End-to-end test with cmsRun PhysicsTools/PatFromScouting/test/test_scoutingToMiniAOD_cfg.py on Run2024C ScoutingPFRun3 data
  • Full cmsDriver chain validated on 1000 events:
    • Step 1: cmsDriver.py scoutMini --step PAT:@scout --era Run3_2024 --conditions auto:run3_data_prompt -n 1000
    • Step 2: cmsDriver.py scoutNano --step NANO:@ScoutMini --era Run3_2024 --conditions auto:run3_data_prompt -n 1000

drkovalskyi and others added 27 commits February 18, 2026 21:17
  Add a constructor from Run3ScoutingElectron that populates kinematics,
  shower shapes, ID variables, track parameters, isolation, and ECAL
  crystal-level information. Scouting-derived quantities are stored as
  userFloats/userInts where GsfElectron members are not directly writable,
  and as dedicated private members for track and ECAL data.

  Provide universal accessors (trkEta, trkPhi, trkpMode, trketaMode,
  trkphiMode, trkqoverpModeError, seedId, nClusters, nCrystals) that
  dispatch to gsfTrack()/superCluster() for standard electrons and to
  the stored scouting values for scouting electrons, controlled by an
  isScoutingElectron() flag. Scouting-only quantities with no standard
  equivalent (energyMatrix, timingMatrix, detIds, rechitZeroSuppression)
  keep their scouting-prefixed accessors.
Add a constructor from Run3ScoutingPhoton that populates kinematics,
shower shapes, energy variables, and isolation. Scouting-derived
quantities are stored as userFloats where reco::Photon members are not
directly writable, and as dedicated private members for ECAL crystal
data.

Provide universal accessors (seedId, nClusters, nCrystals) that
dispatch to superCluster() for standard photons and to stored scouting
values for scouting photons, controlled by an isScoutingPhoton() flag.
Scouting-only quantities with no standard equivalent (energyMatrix,
timingMatrix, detIds, rechitZeroSuppression) keep their scouting-
prefixed accessors.
… since scouting jets don't have constituent indices filled. Still can be useful in the future.
@drkovalskyi
Copy link
Copy Markdown
Contributor Author

@ftorrresd, @battibass, how can we proceed with the PR? We would like to start integrating it with BPHNano and it will be cleaner to keep the two as separate PRs if possible.

@mandrenguyen
Copy link
Copy Markdown
Contributor

please test with cms-sw/cms-bot#2712
Let's see if that works

@mandrenguyen
Copy link
Copy Markdown
Contributor

@drkovalskyi The code-checks looks unrelated to the addition of the new package. Can you run the usual scram b code-checks and scram b code-format?

@cmsbuild
Copy link
Copy Markdown
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-50307/48791

@cmsbuild
Copy link
Copy Markdown
Contributor

Pull request #50307 was updated. @Moanwar, @battibass, @cmsbuild, @davidlange6, @fabiocos, @ftenchini, @ftorrresd, @jfernan2, @mandrenguyen, @srimanob can you please check and sign again.

@mandrenguyen
Copy link
Copy Markdown
Contributor

please test

@cmsbuild
Copy link
Copy Markdown
Contributor

cmsbuild commented Apr 1, 2026

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d2c87f/52381/summary.html
COMMIT: fa4c928
CMSSW: CMSSW_16_1_X_2026-03-31-2300/el8_amd64_gcc13
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/50307/52381/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 5 differences found in the comparisons
  • DQMHistoTests: Total files compared: 53
  • DQMHistoTests: Total histograms compared: 4180749
  • DQMHistoTests: Total failures: 67
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4180662
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 52 files compared)
  • Checked 227 log files, 197 edm output root files, 53 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

workflow id kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
NANOmc106Xul16v2 2500.0001 3.094 3.094 0.000 ( +0.0% ) 3.69 3.80 -3.0% 2.515 2.701
NANOmc106Xul16APVv2 2500.0002 3.048 3.048 0.000 ( +0.0% ) 3.53 3.61 -2.3% 2.562 2.733
NANOmc106Xul17v2 2500.0003 3.153 3.153 0.000 ( +0.0% ) 3.32 3.57 -6.9% 2.928 3.140
NANOmc106Xul18v2 2500.0004 3.105 3.105 0.000 ( +0.0% ) 3.49 3.60 -3.3% 2.928 3.187
NANOdata106Xul16v2 2500.0101 1.540 1.540 0.000 ( +0.0% ) 6.22 6.32 -1.6% 2.637 2.675
NANOdata106Xul17v2 2500.0102 1.926 1.926 0.000 ( +0.0% ) 4.12 4.21 -2.0% 2.870 2.974
NANOdata106Xul18v2 2500.0103 1.890 1.890 0.000 ( +0.0% ) 4.95 5.08 -2.5% 2.728 2.754
NANOmcUL16APVreMINI 2500.0201 0.022 0.022 0.000 ( +0.0% ) 1.90 1.90 +0.2% 2.518 2.549
NANOmcUL16reMINI 2500.0202 0.022 0.022 0.000 ( +0.0% ) 1.86 1.85 +0.6% 2.532 2.554
NANOmcUL17reMINI 2500.0203 0.022 0.022 0.000 ( +0.0% ) 1.67 1.73 -3.0% 2.662 3.023
NANOmcUL18reMINI 2500.0204 0.022 0.022 0.000 ( +0.0% ) 1.63 1.66 -1.8% 2.689 3.002
NANOdataUL16APVreMINI 2500.0301 0.035 0.035 0.000 ( +0.0% ) 1.57 1.63 -3.2% 2.590 2.650
NANOdataUL16reMINI 2500.0302 0.036 0.036 0.000 ( +0.0% ) 1.58 1.60 -1.3% 2.508 2.578
NANOdataUL17reMINI 2500.0303 0.037 0.037 0.000 ( +0.1% ) 1.49 1.59 -5.8% 2.615 2.658
NANOdataUL18reMINI 2500.0304 0.036 0.036 0.000 ( +0.0% ) 1.56 1.58 -1.2% 2.596 2.669
NANOmcUL16APVMini2Mini 2500.0401 0.023 0.023 0.000 ( +0.0% ) 2.53 2.61 -3.1% 2.124 2.231
NANOmcUL16Mini2Mini 2500.0402 0.023 0.023 0.000 ( +0.0% ) 2.59 2.63 -1.5% 2.116 2.247
NANOmcUL17Mini2Mini 2500.0403 0.023 0.023 0.000 ( +0.0% ) 2.36 2.44 -3.4% 2.138 2.252
NANOmcUL18Mini2Mini 2500.0404 0.023 0.023 0.000 ( +0.0% ) 2.43 2.53 -3.9% 2.122 2.257
NANOdataUL16Mini2Mini 2500.0501 0.039 0.039 -0.000 ( -0.1% ) 3.55 3.60 -1.3% 2.002 2.042
NANOdataUL17Mini2Mini 2500.0502 0.040 0.040 0.000 ( +0.0% ) 2.61 2.69 -3.0% 2.051 2.110
NANOdataUL18Mini2Mini 2500.0503 0.039 0.039 0.000 ( +0.0% ) 3.09 3.18 -2.8% 2.047 2.109
NANOmc130X 2500.1001 3.228 3.228 0.000 ( +0.0% ) 4.19 4.26 -1.6% 2.891 2.341
NANOdata130Xrun3 2500.1101 1.790 1.790 0.000 ( +0.0% ) 6.38 6.51 -1.9% 2.612 2.632
NANOmc23Mini2Mini 2500.1201 0.022 0.022 0.000 ( +0.0% ) 2.79 2.85 -2.0% 2.009 2.128
NANOdata23Mini2Mini 2500.1301 0.041 0.041 0.000 ( +0.0% ) 3.55 3.55 +0.1% 2.286 2.404
ScoutingNANOdata124Xrun3 2500.1501 0.772 0.772 0.000 ( +0.0% ) 34.02 34.61 -1.7% 1.460 1.477
ScoutingNANOmonitordata124Xrun3 2500.1502 0.862 0.862 0.000 ( +0.0% ) 13.94 14.22 -2.0% 1.389 1.445
ScoutingNANOdata130Xrun3 2500.1503 1.251 1.251 0.000 ( +0.0% ) 25.09 25.37 -1.1% 1.451 1.430
ScoutingNANOmonitordata130Xrun3 2500.1504 1.297 1.297 0.000 ( +0.0% ) 10.87 10.80 +0.6% 1.363 1.334
muPOGNANOmc140X 2500.2201 2.269 2.269 0.000 ( +0.0% ) 5.53 5.70 -3.0% 1.998 2.178
EGMNANOmc140X 2500.2202 3.859 3.859 0.000 ( +0.0% ) 4.02 4.08 -1.5% 2.308 2.356
BTVNANOmc140X 2500.2203 10.475 10.475 0.000 ( +0.0% ) 2.55 2.58 -1.5% 2.499 2.450
jmeNANOmc140X 2500.2204 6.639 6.639 0.000 ( +0.0% ) 1.13 1.16 -3.0% 2.417 2.630
lepTrackInfoNANOmc140X 2500.2206 3.553 3.553 0.000 ( +0.0% ) 4.00 4.08 -2.0% 2.322 2.367
ScoutingNANOmc140X 2500.2207 1.488 1.488 0.000 ( +0.0% ) 14.36 14.51 -1.0% 1.779 1.783
ScoutingNANOwithPromptmc140X 2500.2208 4.461 4.461 0.000 ( +0.0% ) 3.10 3.22 -3.8% 2.664 2.549
muPOGNANO140Xrun3 2500.2301 1.683 1.683 0.000 ( +0.0% ) 7.66 7.74 -1.1% 2.270 2.250
EGMNANOdata140Xrun3 2500.2302 2.623 2.623 0.000 ( +0.0% ) 5.75 5.83 -1.4% 2.583 2.310
BTVNANOdata140Xrun3 2500.2303 5.612 5.612 0.000 ( +0.0% ) 3.68 3.73 -1.4% 2.771 2.480
jmeNANOdata140Xrun3 2500.2304 3.976 3.976 0.000 ( +0.0% ) 1.46 1.48 -1.2% 2.651 2.311
lepTrackInfoNANOdata140Xrun3 2500.2306 2.413 2.413 0.000 ( +0.0% ) 5.82 5.86 -0.7% 2.590 2.325
ScoutingNANOdata140Xrun3 2500.2307 1.101 1.101 0.000 ( +0.0% ) 29.80 30.44 -2.1% 1.458 1.286
ScoutingNANOmonitordata140Xrun3 2500.2308 2.784 2.784 0.000 ( +0.0% ) 4.92 5.13 -4.2% 2.768 2.893
ScoutingNANOmonitorWithPromptdata140Xrun3 2500.2309 2.784 2.784 0.000 ( +0.0% ) 4.95 5.20 -4.8% 2.778 2.828
L1ScoutingSelectionNANOdata140Xrun3 2500.2311 0.081 0.081 0.000 ( +0.0% ) 47.61 49.45 -3.7% 0.683 0.835
L1ScoutingNANOdata140Xrun3 2500.231 0.082 0.082 0.000 ( +0.0% ) 4.10 4.12 -0.6% 0.831 0.835
l1DPGNANO140Xrun3 2500.2401 9.405 9.405 0.000 ( +0.0% ) 6.49 7.38 -12.1% 1.702 1.456
muDPGNANO140Xrun3 2500.2402 10.377 10.377 0.000 ( +0.0% ) 1.62 1.67 -3.1% 1.575 1.669
muDPGNANOBkg140Xrun3 2500.2403 6.401 6.401 0.000 ( +0.0% ) 14.98 15.35 -2.4% 1.167 1.190
hcalDPGNANO140Xrun3 2500.2404 486.015 486.015 0.000 ( +0.0% ) 1.16 1.17 -0.6% 1.573 1.601
hcalDPGCalibNANO140Xrun3 2500.2405 826.418 826.418 0.000 ( +0.0% ) 1.50 1.51 -0.5% 1.561 1.631
hcalDPGMCNANO140Xrun3 2500.2501 645.333 645.333 0.000 ( +0.0% ) 1.66 1.67 -0.4% 1.639 1.629
NANOmc2024reMINI 2500.2601 0.021 0.021 0.000 ( +0.0% ) 1.91 1.94 -1.5% 2.454 2.690
NANOdata2024reMINI 2500.2701 0.036 0.036 0.000 ( +0.0% ) 1.68 1.91 -12.0% 2.400 2.548
NANOmc150X 2500.3001 3.224 3.224 0.000 ( +0.0% ) 17.14 17.95 -4.5% 2.469 2.702
NANOdata150X 2500.3101 2.144 2.144 0.000 ( +0.0% ) 31.94 34.08 -6.3% 1.805 2.251
NANOdata150X 2500.3102 2.630 2.630 0.000 ( +0.0% ) 26.96 28.39 -5.0% 2.131 2.232
muPOGNANOmc150X 2500.3201 2.344 2.344 0.000 ( +0.0% ) 15.66 16.30 -3.9% 2.313 2.366
EGMNANOmc150X 2500.3202 4.118 4.118 0.000 ( +0.0% ) 14.39 14.87 -3.2% 2.428 2.510
BTVNANOmc150X 2500.3203 10.559 10.559 0.000 ( +0.0% ) 4.80 4.96 -3.2% 2.030 2.687
jmeNANOmc150X 2500.3204 6.460 6.460 0.000 ( +0.0% ) 1.30 1.37 -5.5% 2.005 2.592
jmeNANOrePuppimc150X 2500.3205 6.464 6.464 0.000 ( +0.0% ) 1.20 1.30 -7.5% 2.281 2.961
lepTrackInfoNANOmc150X 2500.3206 3.728 3.728 0.000 ( +0.0% ) 14.67 15.35 -4.4% 2.414 2.440
ScoutingNANOmc150X 2500.3207 1.563 1.563 0.000 ( +0.0% ) 18.98 20.15 -5.8% 2.208 2.277
ScoutingNANOwithPromptmc150X 2500.3208 4.781 4.781 0.000 ( +0.0% ) 10.00 10.63 -5.9% 2.670 2.735
BPHNANOmc150X 2500.3209 4.135 4.135 0.000 ( +0.0% ) 15.49 17.00 -8.9% 1.937 2.598
muPOGNANO150Xrun3 2500.3301 1.244 1.244 0.000 ( +0.0% ) 28.29 30.84 -8.3% 1.847 2.288
EGMNANOdata150Xrun3 2500.3302 2.985 2.985 0.000 ( +0.0% ) 24.99 27.03 -7.5% 2.120 2.203
BTVNANOdata150Xrun3 2500.3303 7.971 7.971 0.000 ( +0.0% ) 5.70 6.20 -8.0% 2.054 2.588
jmeNANOdata150Xrun3 2500.3304 4.349 4.349 0.000 ( +0.0% ) 1.54 1.71 -9.6% 1.984 2.584
jmeNANOrePuppidata150Xrun3 2500.3305 4.332 4.332 0.000 ( +0.0% ) 1.34 1.46 -8.0% 2.229 2.605
lepTrackInfoNANOdata150Xrun3 2500.3306 2.625 2.625 0.000 ( +0.0% ) 25.65 29.14 -12.0% 2.114 2.137
ScoutingNANOdata150Xrun3 2500.3307 1.738 1.738 0.000 ( +0.0% ) 30.12 32.46 -7.2% 1.460 1.500
ScoutingNANOmonitordata150Xrun3 2500.3308 1.400 1.400 0.000 ( +0.0% ) 25.57 27.68 -7.6% 2.053 2.133
ScoutingNANOmonitorWithPromptdata150Xrun3 2500.3309 2.602 2.602 0.000 ( +0.0% ) 16.97 18.25 -7.0% 2.414 2.487
L1ScoutingNANOdata150Xrun3 2500.3311 0.078 0.078 0.000 ( +0.0% ) 3.40 3.54 -4.0% 0.833 0.942
L1ScoutingSelectionNANOdata150Xrun3 2500.3312 0.078 0.078 0.000 ( +0.0% ) 25.33 27.19 -6.8% 0.684 0.941
BPHNANOdata150Xrun3 2500.331 2.632 2.632 0.000 ( +0.0% ) 29.46 34.30 -14.1% 2.181 2.501
L1ScoutingNANOdata160Xrun3 2500.4301 0.110 0.110 0.000 ( +0.0% ) 2.07 2.13 -2.9% 0.824 0.927
L1ScoutingSelectionNANOdata160Xrun3 2500.4302 0.113 0.113 0.000 ( +0.0% ) 18.47 19.42 -4.9% 0.870 0.952
TTbarMINIAOD10.6_UL18v2 2500.9001 1.814 1.814 0.000 ( +0.0% ) 31.31 36.33 -13.8% 1.743 1.830
TTbarMINIAOD14.0 2500.9002 1.662 1.662 0.000 ( +0.0% ) 30.46 35.28 -13.7% 1.373 1.419
DYToLL_M-50_13TeV_pythia8 2500.9101 14.992 14.992 0.000 ( +0.0% ) 6.50 7.17 -9.4% 1.038 1.074
DYToll01234Jets_5f_LO_MLM_Madgraph_LHE_13TeV 2500.9102 0.438 0.150 0.289 ( +193.2% ) 2.92 3.89 -25.0% 0.838 0.836
TTbar_Pow_LHE_13TeV 2500.9103 0.110 0.110 0.000 ( +0.0% ) 2.37 2.54 -6.8% 0.830 0.842

NANO Comparison Summary

Summary:

  • You potentially added 1552 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 32
  • DQMHistoTests: Total histograms compared: 43020
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 43020
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 31 files compared)
  • Checked 189 log files, 110 edm output root files, 32 DQM output files
  • TriggerResults: no differences found

@mandrenguyen
Copy link
Copy Markdown
Contributor

@cms-sw/xpog-l2 @cms-sw/reconstruction-l2 Can you please have a look? Would like to get this in before building a pre-release.

@Moanwar
Copy link
Copy Markdown
Contributor

Moanwar commented Apr 2, 2026

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.