Skip to content

Compartmentalize how ProductRegistry is filled #47533

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 12, 2025

Conversation

Dr15Jones
Copy link
Contributor

@Dr15Jones Dr15Jones commented Mar 7, 2025

PR description:

  • constrain where in the code the ProductRegistry has additional non-dropped entries added
  • isolate when in the construction sequence a signal is sent when the ProductRegistry is changed

PR validation:

This is a code refactoring and as such no changes in behavior are expected. Code compiles and all but one framework unit test pass.

The only known failure is TestIOPoolInputNoParentDictionary. When that test run it copies parts of code from CMSSW to a new build area. The problem is it doesn't copy enough code to work with this change (code in DataFormats/Common/test fails to build).

resolves cms-sw/framework-team#1296

To set the helper, a temporary is created and filled and then moved to the ProductRegistry.
This isolates where where the ProductRegistry can have additional products added.
@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2025

cms-bot internal usage

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2025

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47533/43991

Code check has found code style and quality issues which could be resolved by applying following patch(s)

@Dr15Jones
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2025

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47533/43992

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2025

A new Pull Request was created by @Dr15Jones for master.

It involves the following packages:

  • DataFormats/Provenance (core)
  • FWCore/Framework (core)
  • FWCore/Sources (core)
  • FWCore/TestProcessor (core)
  • IOPool/SecondaryInput (core)

@Dr15Jones, @makortel, @smuzaffar can you please review it and eventually sign? Thanks.
@makortel, @missirol, @mmusich, @rovere, @wddgit this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Mar 7, 2025

-1

Failed Tests: Build HeaderConsistency ClangBuild
Size: This PR adds an extra 480KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1b09ca/44853/summary.html
COMMIT: 9eaf910
CMSSW: CMSSW_15_1_X_2025-03-07-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47533/44853/install.sh to create a dev area with all the needed externals and cmssw changes.

Build

I found compilation error when building:

/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -c -DCMS_MICRO_ARCH='x86-64-v3' -DGNU_GCC -D_GNU_SOURCE -DTBB_USE_GLIBCXX_VERSION=120301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DBOOST_MPL_IGNORE_PARENTHESES_WARNING -DCMSSW_GIT_HASH='CMSSW_15_1_X_2025-03-07-1100' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_15_1_X_2025-03-07-1100' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_X_2025-03-07-1100/src -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/cms/coral/CORAL_2_3_21-94d5b66203ab8a015ad76ed29be3e51e/include/LCG -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/pcre/8.43-2d141998cfe5424b8f7aff48035cc2da/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/boost/1.80.0-7642db88b69862429c1c9c9980662789/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/bz2lib/1.0.6-d065ccd79984efc6d4660f410e4c81de/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/clhep/2.4.7.1-d3a3e353d370e701238f7949a0d7909f/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/curl/7.79.0-d8c0d5017cfad573d276c56bd08114b8/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/gsl/2.6-f7574c606b0ce57ff601d3ca9534cd01/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/libuuid/2.34-27ce4c3579b5b1de2808ea9c4cd8ed29/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/lcg/root/6.32.11-575650675fc09042e42e48da8061099a/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/tbb/v2022.0.0-10fdc07bd3d228f4c274203a8cd20b5c/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/xerces-c/3.1.3-c7b88eaa36d0408120f3c29826a04bf6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/xz/5.2.5-6f3f49b07db84e10c9be594a1176c114/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/zlib/1.2.13-d217cdbdd8d586e845e05946de2796be/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/fmt/10.2.1-e35fd1db5eb3abc8ac0452e8ee427196/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/md5/1.0.0-5b594b264e04ae51e893b1d69a797ec6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/OpenBLAS/0.3.27-70a9dd2c9f309171934f13e3003b0540/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/tinyxml2/6.2.0-f99ae2781d074227d47e8a3e7c8ec87e/include -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++20 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v3 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el8_amd64_gcc12/src/Mixing/Base/src/MixingBase/SecondaryEventProvider.cc.d src/Mixing/Base/src/SecondaryEventProvider.cc -o tmp/el8_amd64_gcc12/src/Mixing/Base/src/MixingBase/SecondaryEventProvider.cc.o
>> Compiling  src/Mixing/Base/src/PileupRandomNumberGenerator.cc
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -c -DCMS_MICRO_ARCH='x86-64-v3' -DGNU_GCC -D_GNU_SOURCE -DTBB_USE_GLIBCXX_VERSION=120301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DBOOST_MPL_IGNORE_PARENTHESES_WARNING -DCMSSW_GIT_HASH='CMSSW_15_1_X_2025-03-07-1100' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_15_1_X_2025-03-07-1100' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/cms/cmssw/CMSSW_15_1_X_2025-03-07-1100/src -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/cms/coral/CORAL_2_3_21-94d5b66203ab8a015ad76ed29be3e51e/include/LCG -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/pcre/8.43-2d141998cfe5424b8f7aff48035cc2da/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/boost/1.80.0-7642db88b69862429c1c9c9980662789/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/bz2lib/1.0.6-d065ccd79984efc6d4660f410e4c81de/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/clhep/2.4.7.1-d3a3e353d370e701238f7949a0d7909f/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/curl/7.79.0-d8c0d5017cfad573d276c56bd08114b8/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/gsl/2.6-f7574c606b0ce57ff601d3ca9534cd01/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/libuuid/2.34-27ce4c3579b5b1de2808ea9c4cd8ed29/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/lcg/root/6.32.11-575650675fc09042e42e48da8061099a/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/tbb/v2022.0.0-10fdc07bd3d228f4c274203a8cd20b5c/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/xerces-c/3.1.3-c7b88eaa36d0408120f3c29826a04bf6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/xz/5.2.5-6f3f49b07db84e10c9be594a1176c114/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/zlib/1.2.13-d217cdbdd8d586e845e05946de2796be/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/fmt/10.2.1-e35fd1db5eb3abc8ac0452e8ee427196/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/md5/1.0.0-5b594b264e04ae51e893b1d69a797ec6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/OpenBLAS/0.3.27-70a9dd2c9f309171934f13e3003b0540/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02879/el8_amd64_gcc12/external/tinyxml2/6.2.0-f99ae2781d074227d47e8a3e7c8ec87e/include -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++20 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v3 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el8_amd64_gcc12/src/Mixing/Base/src/MixingBase/PileupRandomNumberGenerator.cc.d src/Mixing/Base/src/PileupRandomNumberGenerator.cc -o tmp/el8_amd64_gcc12/src/Mixing/Base/src/MixingBase/PileupRandomNumberGenerator.cc.o
In file included from src/Mixing/Base/interface/PileUp.h:14,
                 from src/Mixing/Base/src/PileUp.cc:1:
poison/FWCore/Framework/interface/SignallingProductRegistry.h:1:2: error: #error THIS FILE HAS BEEN REMOVED FROM THE PACKAGE.
    1 | #error THIS FILE HAS BEEN REMOVED FROM THE PACKAGE.
      |  ^~~~~
In file included from src/Mixing/Base/src/PileUp.cc:8:
poison/FWCore/Framework/interface/SignallingProductRegistry.h:1:2: error: #error THIS FILE HAS BEEN REMOVED FROM THE PACKAGE.
    1 | #error THIS FILE HAS BEEN REMOVED FROM THE PACKAGE.


Clang Build

I found compilation warning while trying to compile with clang. Command used:

USER_CUDA_FLAGS='--expt-relaxed-constexpr' USER_CXXFLAGS='-Wno-register -fsyntax-only' /usr/bin/time -v scram build -k -j 32 COMPILER='llvm compile'

See details on the summary page.

@cmsbuild
Copy link
Contributor

-1

Failed Tests: UnitTests
Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1b09ca/44890/summary.html
COMMIT: e2915ba
CMSSW: CMSSW_15_1_X_2025-03-10-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/47533/44890/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test TestIOPoolInputNoParentDictionary had ERRORS

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 5 differences found in the comparisons
  • DQMHistoTests: Total files compared: 49
  • DQMHistoTests: Total histograms compared: 3920300
  • DQMHistoTests: Total failures: 48
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3920232
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
  • Checked 214 log files, 184 edm output root files, 49 DQM output files
  • TriggerResults: no differences found

@Dr15Jones
Copy link
Contributor Author

ignore tests-rejected with manual-override

@civanch
Copy link
Contributor

civanch commented Mar 11, 2025

+1

@Dr15Jones
Copy link
Contributor Author

ping @cms-sw/dqm-l2 ping

@makortel
Copy link
Contributor

Comparison differences are related to #47071

@antoniovagnerini
Copy link

+dqm

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (test failures were overridden). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @mandrenguyen, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

@Dr15Jones
Copy link
Contributor Author

@cms-sw/orp-l2 please merge as this is needed for work I'm presently doing.

@antoniovilela
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit a69a71c into cms-sw:master Mar 12, 2025
10 of 11 checks passed
@Dr15Jones Dr15Jones deleted the limitAddingToProductRegistry branch March 12, 2025 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Compartmentalize how ProductRegistry is filled
6 participants