Skip to content

Move inspectNanoFile.py from test to scripts #47884

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 23, 2025
Merged

Move inspectNanoFile.py from test to scripts #47884

merged 1 commit into from
Apr 23, 2025

Conversation

patinkaew
Copy link
Contributor

@patinkaew patinkaew commented Apr 16, 2025 •
edited
Loading

PR description:

Move inspectNanoFile.py from test (PhysicsTools/NanoAOD/test/inspectNanoFile.py) to scripts (PhysicsTools/NanoAOD/scripts/inspectNanoFile.py) so it is copied to bin, allowing running this as executable without the need to add package (git cms-addpkg PhysicsTools/NanoAOD) first.

PR validation:

Pass scram b runtests use-ibeos.

Pass    6s ... PhysicsTools/NanoAOD/testPhysicsToolsNanoAODTP

After compiling, it is possible to run inspectNanoFile.py under CMSSW workspace as executable as expected:

inspectNanoFile.py -d EventContent.html -s SizeReport.html rootFile.root

The above commands give reasonable EventContent.html, SizeReport.html for data and MC NanoAOD files.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This is not a backport. There is currently no plan for a backport. This is not needed for data-taking.

@cmsbuild cmsbuild added this to the CMSSW_15_1_X milestone Apr 16, 2025
@cmsbuild
Copy link
Contributor

cmsbuild commented Apr 16, 2025 •
edited
Loading

cms-bot internal usage

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47884/44526

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @patinkaew for master.

It involves the following packages:

  • PhysicsTools/NanoAOD (xpog)

@cmsbuild, @ftorrresd, @hqucms can you please review it and eventually sign? Thanks.
@gpetruc this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@ftorrresd
Copy link
Contributor

enable nano

@ftorrresd
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-297fd6/45591/summary.html
COMMIT: 8e47ff3
CMSSW: CMSSW_15_1_X_2025-04-16-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47884/45591/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 10 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3916361
  • DQMHistoTests: Total failures: 17
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3916324
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

Nano size comparison Summary:

  • No valid nano workflows found

@ftorrresd
Copy link
Contributor

please test

"No valid nano workflows found" ??? Not sure I understood. Trying again, just in case...

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-297fd6/45639/summary.html
COMMIT: 8e47ff3
CMSSW: CMSSW_15_1_X_2025-04-18-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47884/45639/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 3 lines to the logs
  • Reco comparison results: 5 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3916361
  • DQMHistoTests: Total failures: 3
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3916338
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

Nano size comparison Summary:

  • No valid nano workflows found

@patinkaew
Copy link
Contributor Author

We got "No valid nano workflows found" again. The reason might be that this PR actually doesn't modify any config file used in NanoAOD workflows, e.g. "*_cff.py".

@iarspider
Copy link
Contributor

iarspider commented Apr 18, 2025 •
edited
Loading

No, the reason is different - the bot doesn't know that the script was moved and thus can't run it. Let me see if I can fix this...

@iarspider iarspider mentioned this pull request Apr 18, 2025
Merged
@iarspider
Copy link
Contributor

please test with cms-sw/cms-bot#2482

@patinkaew
Copy link
Contributor Author

@iarspider, I see. Thank you for your help!

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-297fd6/45641/summary.html
COMMIT: 8e47ff3
CMSSW: CMSSW_15_1_X_2025-04-18-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47884/45641/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 4 lines to the logs
  • Reco comparison results: 5 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3916361
  • DQMHistoTests: Total failures: 44
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3916297
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.001 3.115 3.115 0.000 ( +0.0% ) 6.87 7.02 -2.1% 2.575 2.582
2500.002 3.231 3.231 0.000 ( +0.0% ) 6.11 6.15 -0.7% 3.015 3.006
2500.003 3.172 3.172 0.000 ( +0.0% ) 6.38 6.43 -0.8% 2.998 2.983
2500.011 1.647 1.647 0.000 ( +0.0% ) 11.64 11.87 -1.9% 2.657 2.666
2500.012 2.185 2.185 0.000 ( +0.0% ) 6.41 6.50 -1.4% 2.853 2.853
2500.013 2.002 2.002 0.000 ( +0.0% ) 9.05 9.21 -1.7% 2.747 2.757
2500.021 0.022 0.022 0.000 ( +0.0% ) 1.88 1.96 -4.4% 2.705 2.696
2500.022 0.022 0.022 0.000 ( +0.0% ) 1.82 1.86 -2.2% 2.704 2.699
2500.023 0.022 0.022 0.000 ( +0.0% ) 1.77 1.81 -2.4% 2.563 2.569
2500.024 0.022 0.022 0.000 ( +0.0% ) 1.46 1.50 -3.2% 2.810 2.803
2500.031 0.035 0.035 0.000 ( +0.0% ) 1.65 1.68 -2.0% 2.771 2.753
2500.032 0.036 0.036 0.000 ( +0.0% ) 1.62 1.68 -3.2% 2.727 2.727
2500.033 0.037 0.037 0.000 ( +0.0% ) 1.58 1.62 -2.6% 2.798 2.813
2500.034 0.036 0.036 0.000 ( +0.0% ) 1.58 1.61 -2.2% 2.790 2.787
2500.101 2.872 2.872 0.000 ( +0.0% ) 15.83 16.31 -2.9% 2.650 2.643
2500.111 1.474 1.474 0.000 ( +0.0% ) 31.33 32.25 -2.9% 2.336 2.322
2500.112 1.896 1.896 0.000 ( +0.0% ) 26.69 26.55 +0.5% 2.407 2.411
2500.131 0.750 0.750 0.000 ( +0.0% ) 37.67 37.81 -0.4% 1.496 1.502
2500.201 2.706 2.706 0.000 ( +0.0% ) 13.25 13.56 -2.3% 2.210 2.214
2500.211 1.845 1.845 0.000 ( +0.0% ) 27.27 27.69 -1.5% 2.409 2.413
2500.212 2.243 2.243 0.000 ( +0.0% ) 23.06 23.45 -1.7% 2.498 2.500
2500.221 2.141 2.141 0.000 ( +0.0% ) 14.06 14.10 -0.3% 2.135 2.138
2500.222 3.516 3.516 0.000 ( +0.0% ) 13.41 13.46 -0.3% 2.219 2.226
2500.223 10.328 10.328 0.000 ( +0.0% ) 4.76 4.75 +0.3% 2.393 2.386
2500.224 6.622 6.622 0.000 ( +0.0% ) 1.25 1.28 -1.7% 2.299 2.300
2500.225 6.671 6.671 0.000 ( +0.0% ) 1.18 1.19 -1.1% 2.500 2.512
2500.226 3.210 3.210 0.000 ( +0.0% ) 13.78 14.07 -2.1% 2.220 2.212
2500.227 1.442 1.442 0.000 ( +0.0% ) 24.04 24.15 -0.5% 1.442 1.443
2500.228 4.045 4.045 0.000 ( +0.0% ) 8.87 8.97 -1.2% 2.254 2.315
2500.231 1.516 1.516 0.000 ( +0.0% ) 22.98 23.26 -1.2% 2.041 2.313
2500.232 2.502 2.502 0.000 ( +0.0% ) 21.38 22.03 -2.9% 2.187 2.409
2500.233 5.422 5.422 0.000 ( +0.0% ) 7.04 7.17 -1.7% 2.557 2.567
2500.234 3.928 3.928 0.000 ( +0.0% ) 1.56 1.63 -3.7% 2.484 2.244
2500.235 3.960 3.960 0.000 ( +0.0% ) 1.50 1.53 -1.9% 2.682 2.434
2500.236 2.292 2.292 0.000 ( +0.0% ) 22.61 23.04 -1.9% 2.398 2.403
2500.237 1.018 1.018 0.000 ( +0.0% ) 34.57 35.40 -2.3% 1.453 1.451
2500.238 2.477 2.477 0.000 ( +0.0% ) 16.49 17.15 -3.8% 2.469 2.459
2500.241 9.404 9.404 0.000 ( +0.0% ) 7.69 7.10 +8.4% 1.933 1.931
2500.242 10.331 10.331 0.000 ( +0.0% ) 1.69 1.67 +1.2% 1.726 1.729
2500.243 2.712 2.712 0.000 ( +0.0% ) 16.52 16.59 -0.4% 1.061 1.064
2500.244 486.016 486.016 0.000 ( +0.0% ) 1.14 1.16 -1.7% 1.736 1.691
2500.245 826.413 826.413 0.000 ( +0.0% ) 1.51 1.54 -1.6% 1.692 1.676
2500.251 645.333 645.333 0.000 ( +0.0% ) 1.63 1.65 -1.3% 1.781 1.788
2500.301 0.021 0.021 0.000 ( +0.0% ) 1.74 1.79 -2.5% 2.821 2.821
2500.311 0.036 0.036 0.000 ( +0.0% ) 1.68 1.69 -1.0% 2.772 2.757
2500.901 1.819 1.819 0.000 ( +0.0% ) 44.82 47.57 -5.8% 1.437 1.437
2500.902 1.665 1.665 0.000 ( +0.0% ) 47.20 47.36 -0.4% 1.338 1.341
2500.911 14.345 14.345 0.000 ( +0.0% ) 8.80 8.72 +1.0% 1.092 1.091
2500.912 0.199 0.199 0.000 ( +0.0% ) 3.13 3.02 +3.5% 0.847 0.850
2500.913 0.110 0.110 0.000 ( +0.0% ) 2.53 2.55 -0.9% 0.852 0.848

@ftorrresd
Copy link
Contributor

+1

@patinkaew @iarspider

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @rappoccio, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2)
Notice This PR was tested with additional Pull Request(s), please also merge them if necessary: cms-sw/cms-bot#2482

@mandrenguyen
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 8885be2 into cms-sw:master Apr 23, 2025
13 checks passed
@cmsbuild cmsbuild mentioned this pull request Apr 23, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
code-checks-approved fully-signed orp-approved requires-external tests-approved xpog-approved
Projects
None yet
Milestone
CMSSW_15_1_X
Development

Successfully merging this pull request may close these issues.
5 participants
@patinkaew @cmsbuild @ftorrresd @iarspider @mandrenguyen