Skip to content

no delay read when runing PAT or NANO by itself [150X] #47899

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Apr 23, 2025
Merged

no delay read when runing PAT or NANO by itself [150X] #47899

merged 3 commits into from
Apr 23, 2025

Conversation

vlimant
Copy link
Contributor

@vlimant vlimant commented Apr 17, 2025 •
edited
Loading

PR description:

backport of #47897

as a follow up of #47750 we can disable the delayed read when running PAT or NANO as the first step

@cmsbuild cmsbuild added this to the CMSSW_15_0_X milestone Apr 17, 2025
@cmsbuild
Copy link
Contributor

A new Pull Request was created by @vlimant for CMSSW_15_0_X.

It involves the following packages:

  • Configuration/Applications (operations)

@antoniovilela, @cmsbuild, @davidlange6, @fabiocos, @mandrenguyen, @rappoccio can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @fabiocos, @makortel, @missirol this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 17, 2025 •
edited
Loading

cms-bot internal usage

@vlimant
Copy link
Contributor Author

vlimant commented Apr 17, 2025

assign xpog

@cmsbuild
Copy link
Contributor

New categories assigned: xpog

@ftorrresd,@hqucms you have been requested to review this Pull request/Issue and eventually sign? Thanks

@vlimant
Copy link
Contributor Author

vlimant commented Apr 17, 2025

enable nano

@vlimant
Copy link
Contributor Author

vlimant commented Apr 17, 2025

please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 44KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-05f303/45609/summary.html
COMMIT: 23873de
CMSSW: CMSSW_15_0_X_2025-04-16-2300/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47899/45609/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

NANO Comparison Summary

Summary:

  • You potentially removed 71 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 23
  • DQMHistoTests: Total histograms compared: 90269
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 90269
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 22 files compared)
  • Checked 113 log files, 65 edm output root files, 23 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.001 3.115 3.115 0.000 ( +0.0% ) 6.80 5.10 +33.3% 2.574 2.588
2500.002 3.231 3.231 0.000 ( +0.0% ) 6.08 4.55 +33.6% 3.011 3.013
2500.003 3.172 3.172 0.000 ( +0.0% ) 6.31 4.78 +32.2% 2.980 2.988
2500.011 1.647 1.647 0.000 ( +0.0% ) 11.44 8.96 +27.7% 2.648 2.666
2500.012 2.185 2.185 0.000 ( +0.0% ) 6.38 4.86 +31.2% 2.841 2.852
2500.013 2.002 2.002 0.000 ( +0.0% ) 8.99 6.86 +31.0% 2.750 2.749
2500.021 0.022 0.022 0.000 ( +0.0% ) 1.86 1.30 +42.6% 2.951 2.699
2500.022 0.022 0.022 0.000 ( +0.0% ) 1.77 1.24 +42.9% 2.947 2.705
2500.023 0.022 0.022 0.000 ( +0.0% ) 1.57 1.17 +34.5% 2.667 2.571
2500.024 0.022 0.022 0.000 ( +0.0% ) 1.40 1.00 +39.9% 3.074 2.793
2500.031 0.035 0.035 0.000 ( +0.0% ) 1.59 1.06 +49.6% 2.888 2.768
2500.032 0.036 0.036 0.000 ( +0.0% ) 1.55 1.06 +46.2% 2.890 2.723
2500.033 0.037 0.037 0.000 ( +0.0% ) 1.55 1.04 +49.5% 2.930 2.802
2500.034 0.036 0.036 0.000 ( +0.0% ) 1.51 1.04 +46.0% 2.917 2.785
2500.101 2.872 2.872 0.000 ( +0.0% ) 15.45 11.27 +37.1% 2.640 2.644
2500.111 1.474 1.474 0.000 ( +0.0% ) 29.98 20.52 +46.1% 2.326 2.342
2500.112 1.896 1.896 0.000 ( +0.0% ) 25.40 18.43 +37.8% 2.406 2.406
2500.131 0.750 0.750 0.000 ( +0.0% ) 36.34 25.47 +42.7% 1.477 1.476
2500.201 2.706 2.706 0.000 ( +0.0% ) 11.92 8.99 +32.6% 2.208 2.212
2500.211 1.845 1.845 0.000 ( +0.0% ) 25.75 19.46 +32.3% 2.404 2.401
2500.212 2.243 2.243 0.000 ( +0.0% ) 22.49 14.73 +52.7% 2.493 2.493
2500.221 2.141 2.141 0.000 ( +0.0% ) 13.48 8.20 +64.4% 2.128 2.124
2500.222 3.516 3.516 0.000 ( +0.0% ) 12.89 10.08 +27.9% 2.222 2.224
2500.223 10.328 10.328 0.000 ( +0.0% ) 4.55 3.02 +50.8% 2.316 2.324
2500.224 6.622 6.622 0.000 ( +0.0% ) 1.24 0.98 +26.5% 2.294 2.210
2500.225 6.671 6.671 0.000 ( +0.0% ) 1.16 0.88 +31.2% 2.478 2.130
2500.226 3.210 3.210 0.000 ( +0.0% ) 13.30 8.81 +50.9% 2.212 2.208
2500.227 1.442 1.442 0.000 ( +0.0% ) 23.04 19.13 +20.4% 1.441 1.430
2500.228 4.045 4.045 0.000 ( +0.0% ) 8.41 6.37 +32.1% 2.213 2.306
2500.231 1.516 1.516 0.000 ( +0.0% ) 22.04 14.56 +51.3% 2.311 2.302
2500.232 2.502 2.502 0.000 ( +0.0% ) 20.84 15.61 +33.5% 2.401 2.397
2500.233 5.422 5.422 0.000 ( +0.0% ) 6.75 6.30 +7.1% 2.553 2.575
2500.234 3.928 3.928 0.000 ( +0.0% ) 1.58 1.22 +29.7% 2.485 2.242
2500.235 3.960 3.960 0.000 ( +0.0% ) 1.50 1.14 +31.0% 2.675 2.424
2500.236 2.292 2.292 0.000 ( +0.0% ) 21.87 20.11 +8.7% 2.394 2.394
2500.237 1.018 1.018 0.000 ( +0.0% ) 34.28 28.42 +20.6% 1.450 1.454
2500.238 2.477 2.477 0.000 ( +0.0% ) 17.13 12.18 +40.6% 2.471 2.451
2500.241 9.404 9.404 0.000 ( +0.0% ) 7.31 6.43 +13.6% 1.928 1.925
2500.242 10.331 10.331 0.000 ( +0.0% ) 1.65 1.49 +10.8% 1.725 1.724
2500.243 2.712 2.712 0.000 ( +0.0% ) 15.67 11.50 +36.2% 1.058 1.061
2500.244 486.016 486.016 0.000 ( +0.0% ) 1.13 0.72 +56.0% 1.685 1.679
2500.245 826.413 826.413 0.000 ( +0.0% ) 1.49 1.02 +46.6% 1.668 1.685
2500.251 645.333 645.333 0.000 ( +0.0% ) 1.63 1.14 +42.9% 1.784 1.771
2500.301 0.021 0.021 0.000 ( +0.0% ) 1.68 1.11 +51.5% 3.100 2.816
2500.311 0.036 0.036 0.000 ( +0.0% ) 1.67 1.09 +53.5% 2.893 2.771
2500.901 1.819 1.819 0.000 ( +0.0% ) 42.69 34.36 +24.3% 1.432 1.432
2500.902 1.665 1.665 0.000 ( +0.0% ) 47.06 28.64 +64.4% 1.333 1.333
2500.911 14.345 14.345 0.000 ( +0.0% ) 8.18 5.55 +47.5% 1.083 1.089
2500.912 0.150 0.438 -0.288 ( -65.8% ) 2.73 1.67 +63.1% 0.849 0.845
2500.913 0.110 0.110 0.000 ( +0.0% ) 2.55 1.53 +66.4% 0.850 0.849

@vlimant vlimant mentioned this pull request Apr 17, 2025
Open
@cmsbuild
Copy link
Contributor

Pull request #47899 was updated. @antoniovilela, @cmsbuild, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @mandrenguyen, @rappoccio can you please check and sign again.

@vlimant
Copy link
Contributor Author

vlimant commented Apr 18, 2025

please test

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals-NANO
Size: This PR adds an extra 44KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-05f303/45630/summary.html
COMMIT: f5db5b8
CMSSW: CMSSW_15_0_X_2025-04-17-2300/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47899/45630/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-NANO

  • 2500.2382500.238_ScoutingNANOwithPromptdata140Xrun3/step2_ScoutingNANOwithPromptdata140Xrun3.log

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 233 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4007714
  • DQMHistoTests: Total failures: 4081
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4003613
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 218 log files, 189 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@cmsbuild
Copy link
Contributor

Pull request #47899 was updated. @antoniovilela, @cmsbuild, @davidlange6, @fabiocos, @ftorrresd, @hqucms, @mandrenguyen, @rappoccio can you please check and sign again.

@vlimant
Copy link
Contributor Author

vlimant commented Apr 18, 2025

please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 44KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-05f303/45634/summary.html
COMMIT: e57ee06
CMSSW: CMSSW_15_0_X_2025-04-18-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47899/45634/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 1 lines to the logs
  • Reco comparison results: 8 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4007714
  • DQMHistoTests: Total failures: 124
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4007570
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 218 log files, 189 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially removed 1185 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 23
  • DQMHistoTests: Total histograms compared: 90269
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 90269
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 22 files compared)
  • Checked 113 log files, 65 edm output root files, 23 DQM output files
  • TriggerResults: no differences found

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.001 3.115 3.115 0.000 ( +0.0% ) 6.97 6.99 -0.2% 2.591 2.584
2500.002 3.231 3.231 0.000 ( +0.0% ) 6.21 6.20 +0.2% 3.026 3.001
2500.003 3.172 3.172 0.000 ( +0.0% ) 6.42 6.47 -0.8% 3.037 2.979
2500.011 1.647 1.647 0.000 ( +0.0% ) 11.78 11.87 -0.8% 2.685 2.667
2500.012 2.185 2.185 0.000 ( +0.0% ) 6.49 6.47 +0.2% 2.864 2.849
2500.013 2.002 2.002 0.000 ( +0.0% ) 9.24 9.26 -0.3% 2.826 2.758
2500.021 0.022 0.022 0.000 ( +0.0% ) 1.92 1.93 -0.6% 2.948 2.705
2500.022 0.022 0.022 0.000 ( +0.0% ) 1.83 1.87 -2.2% 2.947 2.703
2500.023 0.022 0.022 0.000 ( +0.0% ) 1.63 1.80 -9.7% 2.669 2.562
2500.024 0.022 0.022 0.000 ( +0.0% ) 1.46 1.50 -2.5% 3.067 2.813
2500.031 0.035 0.035 0.000 ( +0.0% ) 1.65 1.66 -0.9% 2.904 2.775
2500.032 0.036 0.036 0.000 ( +0.0% ) 1.66 1.71 -2.9% 2.894 2.721
2500.033 0.037 0.037 0.000 ( +0.0% ) 1.57 1.61 -2.4% 2.952 2.807
2500.034 0.036 0.036 0.000 ( +0.0% ) 1.59 1.63 -2.2% 2.944 2.789
2500.101 2.872 2.872 0.000 ( +0.0% ) 16.02 16.27 -1.5% 2.693 2.643
2500.111 1.474 1.474 0.000 ( +0.0% ) 30.70 31.85 -3.6% 2.402 2.337
2500.112 1.896 1.896 0.000 ( +0.0% ) 25.22 26.01 -3.0% 2.027 2.408
2500.131 0.750 0.750 0.000 ( +0.0% ) 37.54 37.59 -0.1% 1.421 1.392
2500.201 2.706 2.706 0.000 ( +0.0% ) 14.14 13.66 +3.5% 1.974 2.154
2500.211 1.845 1.845 0.000 ( +0.0% ) 26.10 27.24 -4.2% 1.989 2.110
2500.212 2.243 2.243 0.000 ( +0.0% ) 21.96 22.60 -2.8% 2.223 2.220
2500.221 2.141 2.141 0.000 ( +0.0% ) 13.76 14.14 -2.7% 1.772 2.077
2500.222 3.516 3.516 0.000 ( +0.0% ) 13.07 13.30 -1.7% 1.717 2.161
2500.223 10.328 10.328 0.000 ( +0.0% ) 4.66 4.73 -1.5% 2.012 2.227
2500.224 6.622 6.622 0.000 ( +0.0% ) 1.27 1.28 -1.1% 2.316 2.346
2500.225 6.671 6.671 0.000 ( +0.0% ) 1.19 1.18 +0.9% 2.537 2.546
2500.226 3.210 3.210 0.000 ( +0.0% ) 13.54 13.95 -2.9% 2.182 2.214
2500.227 1.442 1.442 0.000 ( +0.0% ) 19.12 24.18 -20.9% 1.790 1.443
2500.228 4.045 4.045 0.000 ( +0.0% ) 9.31 8.93 +4.3% 2.333 2.322
2500.231 1.516 1.516 0.000 ( +0.0% ) 21.72 23.54 -7.7% 2.428 2.309
2500.232 2.502 2.502 0.000 ( +0.0% ) 20.86 22.01 -5.2% 2.512 2.402
2500.233 5.422 5.422 0.000 ( +0.0% ) 7.08 7.21 -1.7% 2.666 2.567
2500.234 3.928 3.928 0.000 ( +0.0% ) 1.63 1.63 +0.0% 2.623 2.484
2500.235 3.960 3.960 0.000 ( +0.0% ) 1.54 1.54 -0.1% 2.811 2.680
2500.236 2.292 2.292 0.000 ( +0.0% ) 22.44 23.22 -3.4% 2.505 2.395
2500.237 1.018 1.018 0.000 ( +0.0% ) 35.53 35.48 +0.1% 1.505 1.454
2500.238 2.477 2.477 0.000 ( +0.0% ) 17.13 17.22 -0.5% 2.569 2.474
2500.241 9.404 9.404 0.000 ( +0.0% ) 7.10 7.34 -3.2% 1.929 1.930
2500.242 10.331 10.331 0.000 ( +0.0% ) 1.70 1.66 +2.5% 1.723 1.725
2500.243 2.712 2.712 0.000 ( +0.0% ) 16.86 16.48 +2.3% 1.060 1.058
2500.244 486.016 486.016 0.000 ( +0.0% ) 1.15 1.16 -0.4% 1.680 1.689
2500.245 826.413 826.413 0.000 ( +0.0% ) 1.55 1.54 +0.7% 1.663 1.679
2500.251 645.333 645.333 0.000 ( +0.0% ) 1.66 1.67 -0.6% 1.781 1.778
2500.301 0.021 0.021 0.000 ( +0.0% ) 1.76 1.75 +1.0% 3.097 2.813
2500.311 0.036 0.036 0.000 ( +0.0% ) 1.70 1.69 +0.2% 2.889 2.763
2500.901 1.819 1.819 0.000 ( +0.0% ) 34.20 43.88 -22.1% 1.873 1.432
2500.902 1.665 1.665 0.000 ( +0.0% ) 33.30 46.22 -27.9% 1.457 1.335
2500.911 14.345 14.345 0.000 ( +0.0% ) 8.88 7.93 +11.9% 1.090 1.090
2500.912 0.171 0.150 0.020 ( +13.6% ) 2.90 2.52 +15.0% 0.849 0.850
2500.913 0.110 0.110 0.000 ( +0.0% ) 2.62 2.63 -0.2% 0.851 0.852

@vlimant vlimant changed the title no delay read when runing PAT by itself [150X] no delay read when runing PAT or NANO by itself [150X] Apr 22, 2025
@vlimant
Copy link
Contributor Author

vlimant commented Apr 22, 2025

If we can have this in the next 15.0 patch release, that'd be great

@ftorrresd
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_15_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_15_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @mandrenguyen, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

@mandrenguyen
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit d45e5e7 into cms-sw:CMSSW_15_0_X Apr 23, 2025
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
fully-signed operations-approved orp-approved tests-approved xpog-approved
Projects
None yet
Milestone
CMSSW_15_0_X
Development

Successfully merging this pull request may close these issues.
4 participants
@vlimant @cmsbuild @ftorrresd @mandrenguyen