Skip to content

Conversation

@mmusich
Copy link
Contributor

@mmusich mmusich commented Aug 8, 2025

PR description:

Title says it all.
When developing a test for CMSHLT-3534 I realized that certain online client configurations customize the event source as if it was a DQMStreamerReader even in the case it which options.inputFiles is not null. In that case as per this code here:

else:
print("The list of input files is provided. Disabling discovery and running on everything.")
files = ["file://" + x for x in options.inputFiles]
source = cms.Source("PoolSource",
fileNames = cms.untracked.vstring(files),
secondaryFileNames = cms.untracked.vstring()
)

the source gets changed to a traditional PoolSource and the customization doesn't make sense, hence leading to runtime errors.
This PR introduces protections to circumvent the issue.

PR validation:

Run successfully the following script in CMSSW_15_0_12 on a file produced with the GRun menu proposed in CMSHLT-3534.

#!/bin/bash

echo "starting to run the DQM tests"

cd CMSSW_15_0_12/src/
# Input ROOT file (adjust if needed)
INPUT_FILE="../../testCMSHLT3534_hlt.root"

# Directory containing the DQM client config files
CFG_DIR="DQM/Integration/python/clients"

# List of client config filenames
CLIENTS=(
    beam_dqm_sourceclient-live_cfg.py
    beampixel_dqm_sourceclient-live_cfg.py
    beamspotdip_dqm_sourceclient-live_cfg.py
    csc_dqm_sourceclient-live_cfg.py
    ctpps_dqm_sourceclient-live_cfg.py
    #dt4ml_dqm_sourceclient-live_cfg.py
    dt_dqm_sourceclient-live_cfg.py
    ecal_dqm_sourceclient-live_cfg.py
    es_dqm_sourceclient-live_cfg.py
    fed_dqm_sourceclient-live_cfg.py
    gem_dqm_sourceclient-live_cfg.py
    hcal_dqm_sourceclient-live_cfg.py
    hcalreco_dqm_sourceclient-live_cfg.py
    hlt_dqm_sourceclient-live_cfg.py
    info_dqm_sourceclient-live_cfg.py
    l1tstage2_dqm_sourceclient-live_cfg.py
    l1tstage2emulator_dqm_sourceclient-live_cfg.py
    mutracking_dqm_sourceclient-live_cfg.py
    onlinebeammonitor_dqm_sourceclient-live_cfg.py
    pixel_dqm_sourceclient-live_cfg.py
    pixellumi_dqm_sourceclient-live_cfg.py
    rpc_dqm_sourceclient-live_cfg.py
    sistrip_dqm_sourceclient-live_cfg.py
    visualization-live_cfg.py
)

# Make a logs directory if not present
mkdir -p logs

for client in "${CLIENTS[@]}"; do
    echo "Running $client..."

    LOGFILE="logs/${client%.py}.log"

    cmsRun "$CFG_DIR/$client" inputFiles="$INPUT_FILE" > "$LOGFILE" 2>&1

    if [ $? -ne 0 ]; then
        echo "$client FAILED. Check $CMSSW_BASE/src/$LOGFILE"
    else
        echo "$client completed successfully."
    fi
done

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

N/A

@mmusich mmusich changed the title protect online DQM clients when running in inputFile mode against meaningless customizations protect online DQM clients when running in inputFiles mode against meaningless customizations Aug 8, 2025
@cmsbuild cmsbuild added this to the CMSSW_15_1_X milestone Aug 8, 2025
@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 8, 2025

cms-bot internal usage

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 8, 2025

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48705/45740

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 8, 2025

A new Pull Request was created by @mmusich for master.

It involves the following packages:

  • DQM/Integration (dqm)

@antoniovagnerini, @cmsbuild, @ctarricone, @rseidita can you please review it and eventually sign? Thanks.
@batinkov, @francescobrivio, @threus this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@mmusich
Copy link
Contributor Author

mmusich commented Aug 8, 2025

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 8, 2025

-1

Failed Tests: RelVals-INPUT
Size: This PR adds an extra 40KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7ac625/47619/summary.html
COMMIT: 452ab1e
CMSSW: CMSSW_15_1_X_2025-08-08-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48705/47619/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

  • 136.897136.897_RunCosmics2021CRUZET/step2_RunCosmics2021CRUZET.log

Comparison Summary

Summary:

  • You potentially removed 2 lines from the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4076122
  • DQMHistoTests: Total failures: 43
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4076059
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@mmusich
Copy link
Contributor Author

mmusich commented Aug 9, 2025

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 9, 2025

+1

Size: This PR adds an extra 40KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7ac625/47638/summary.html
COMMIT: 452ab1e
CMSSW: CMSSW_15_1_X_2025-08-08-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48705/47638/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 3 lines from the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4076122
  • DQMHistoTests: Total failures: 73
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4076029
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@mmusich
Copy link
Contributor Author

mmusich commented Aug 18, 2025

@cms-sw/dqm-l2 do you have comments on this one?

@antoniovagnerini
Copy link

+dqm
-P5 tests successful

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @mandrenguyen, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

@antoniovagnerini
Copy link

@cms-sw/dqm-l2 do you have comments on this one?

no, the fix you proposes to circumvent the issue is ok with us, thanks for catching it

@mandrenguyen
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 46724de into cms-sw:master Aug 21, 2025
10 checks passed
@mmusich mmusich deleted the mm_dev_clients_protect_inputFiles branch August 21, 2025 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants