ECAL Phase 2 - Upgrades for Phase 2 development workflow, validation and DQM

[thomreis]

PR description:

This PR removes legacy modules from the ECAL Phase 2 development workflows and updates the validation to work without endcaps and preshower collections. The workflows also use the proper Phase 2 pulse shape obtained from 2018 TB data.
The EcalLazyClusterTools were updated to make an EE input collection optional.
ECAL DQM tasks and clients not needed for Phase 2 are removed from the workflow. This is not the final setup. Some tasks may be added or modified at a later time when the required inputs become available.

No changes are expected for Run 3 workflows. Phase 2 workflows could have several endcap and preshower DQM and validation plots removed but the barrel ones should not be affected.

PR validation:

Passes limited matrix tests (except some file open errors) as well as WFs 34034.0 and ECAL Phase 2 development workflows 34034.61 and 34034.612.

[cmsbuild]

cms-bot internal usage

[thomreis]

type ecal

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-49409/46864

• There are other open Pull requests which might conflict with changes you have proposed:
  • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Fix location of alpaka modification for particleFlowClusterHCAL #49295, Update mkFit-related processModifier for LST-seeded phase-2 HLT single tracking iteration #49383
  • File Configuration/StandardSequences/python/RawToDigi_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
  • File DQM/EcalMonitorTasks/python/EcalMonitorTask_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076
  • File DQMOffline/Configuration/python/DQMOffline_SecondStep_cff.py modified in PR(s): Tau DQM miniAOD validation for DATA #44029, Update BTV offline DQM sequences #46838
  • File DQMOffline/Configuration/python/DQMOffline_cff.py modified in PR(s): Tau DQM miniAOD validation for DATA #44029, Update BTV offline DQM sequences #46838
  • File EventFilter/EcalRawToDigi/python/ecalDigis_cff.py modified in PR(s): Remove the configuration of the legacy CUDA workflows #46076

[cmsbuild]

A new Pull Request was created by @thomreis for master.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv)
• Configuration/StandardSequences (operations)
• DQM/EcalMonitorClient (dqm)
• DQM/EcalMonitorTasks (dqm)
• DQMOffline/Configuration (dqm)
• DQMOffline/Ecal (dqm)
• EventFilter/EcalRawToDigi (reconstruction)
• RecoEcal/Configuration (reconstruction)
• RecoEcal/EgammaCoreTools (reconstruction)
• Validation/Configuration (dqm, simulation)
• Validation/EcalClusters (dqm)
• Validation/EcalDigis (dqm)
• Validation/EcalHits (dqm)
• Validation/EcalRecHits (dqm)

@AdrianoDee, @DickyChant, @antoniovagnerini, @civanch, @cmsbuild, @ctarricone, @davidlange6, @fabiocos, @ftenchini, @gabrielmscampos, @jfernan2, @kpedro88, @mandrenguyen, @mdhildreth, @miquork, @nothingface0, @rseidita, @srimanob can you please review it and eventually sign? Thanks.
@GiacomoSguazzoni, @Martin-Grunewald, @Prasant1993, @ReyerBand, @Sam-Harper, @VinInn, @VourMa, @a-kapoor, @afiqaize, @apsallid, @argiro, @denizsun, @dgulhan, @elusian, @fabiocos, @felicepantaleo, @jainshilpi, @lgray, @makortel, @missirol, @mmasciov, @mmusich, @mtosi, @ram1123, @rchatter, @rociovilar, @rovere, @salimcerci, @sameasy, @slomeo, @sobhatta, @thomreis, @threus, @valsdav, @varuns23, @wang0jin, @youyingli this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[thomreis]

please test

[cmsbuild]

+1

Size: This PR adds an extra 260KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-107a18/49529/summary.html
COMMIT: 2c63104
CMSSW: CMSSW_16_0_X_2025-11-18-1100/el8_amd64_gcc13
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/49409/49529/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 7 lines from the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 10 differences found in the comparisons
• DQMHistoTests: Total files compared: 51
• DQMHistoTests: Total histograms compared: 3895553
• DQMHistoTests: Total failures: 65
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3895468
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -14731.64 KiB( 50 files compared)
• DQMHistoSizes: changed ( 29634.0,... ): -1843.433 KiB EcalDigisV/SelectiveReadout
• DQMHistoSizes: changed ( 29634.0,... ): -951.648 KiB EcalDigisV/EcalDigiTask
• DQMHistoSizes: changed ( 29634.0,... ): -151.247 KiB EcalHitsV/EcalSimHitsValidation
• Checked 218 log files, 188 edm output root files, 51 DQM output files
• TriggerResults: no differences found

[jfernan2]

+1

[civanch]

+1

[nothingface0]

Naive question: does this affect online DQM as well? I cannot clearly tell from the files touched.

[thomreis]

Naive question: does this affect online DQM as well? I cannot clearly tell from the files touched.

We did not change the online DQM configuration so the Run 3 one should not be changed.
Unless the phase2_ecal_devel modifier is active I do not think there should be changes.
Is there an online DQM for Phase 2 already?

[AdrianoDee]

+pdmv

[nothingface0]

We did not change the online DQM configuration so the Run 3 one should not be changed. Unless the phase2_ecal_devel modifier is active I do not think there should be changes.

Thanks!

Is there an online DQM for Phase 2 already?

Not that I know of!

[nothingface0]

+dqm

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @ftenchini, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

[mandrenguyen]

+1