Skip to content

Relval wf 539.0 segfaults in el8_ppc64le_gcc12 #44059

Open
@aandvalenzuela

Description

@aandvalenzuela

Hello,

Herwig RelVal 539.0` segfaults in el8_ppc64le_gcc12 with the following stacktrace:

Thread 5 (Thread 0x100840f491f0 (LWP 27893) "cmsRun"):
#0  0x0000100002de5b04 in nanosleep () from /lib64/libc.so.6
#1  0x0000100002de5940 in sleep () from /lib64/libc.so.6
#2  0x0000100009ed595c in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  <signal handler called>
#4  0x0000100002e22ac4 in syscall () from /lib64/libc.so.6
#5  0x00001000025be308 in tbb::detail::r1::futex_wait (comparand=2, futex=0x100008c10024) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:100
#6  tbb::detail::r1::binary_semaphore::P (this=0x100008c10024) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:253
#7  tbb::detail::r1::rml::internal::thread_monitor::wait (this=0x100008c10020) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/rml_thread_monitor.h:235
#8  tbb::detail::r1::rml::private_worker::run (this=0x100008c10000) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:273
#9  tbb::detail::r1::rml::private_worker::thread_routine (arg=0x100008c10000) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:221
#10 0x0000100002ca9718 in start_thread () from /lib64/libpthread.so.0
#11 0x0000100002e2b778 in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x1008405391f0 (LWP 27892) "cmsRun"):
#0  0x0000100002e19e70 in poll () from /lib64/libc.so.6
#1  0x0000100009ed63a4 in full_read.constprop () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#2  0x0000100009ed7b38 in edm::service::InitRootHandlers::stacktraceFromThread() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  0x0000100009edb63c in sig_dostack_then_abort () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#4  <signal handler called>
#5  0x0000100848d3e6a0 in Herwig::MatchboxHybridAmplitude::canHandle(std::vector<ThePEG::Pointer::RCPtr<ThePEG::ParticleData>, std::allocator<ThePEG::Pointer::RCPtr<ThePEG::ParticleData> > > const&, ThePEG::Pointer::TransientRCPtr<Herwig::MatchboxFactory>, bool) const () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/external/herwig7/7.2.2-d0cde4fd0acf1e04a2f4dadab541f205/lib/Herwig/Herwig.so.27
#6  0x0000100848cf8998 in Herwig::MatchboxFactory::makeMEs(std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, unsigned int, bool) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/external/herwig7/7.2.2-d0cde4fd0acf1e04a2f4dadab541f205/lib/Herwig/Herwig.so.27
#7  0x0000100848cfdf7c in Herwig::MatchboxFactory::setup() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/external/herwig7/7.2.2-d0cde4fd0acf1e04a2f4dadab541f205/lib/Herwig/Herwig.so.27
#8  0x0000100848cffcd4 in Herwig::MatchboxFactory::doinit() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/external/herwig7/7.2.2-d0cde4fd0acf1e04a2f4dadab541f205/lib/Herwig/Herwig.so.27
#9  0x0000100846e212d4 in ThePEG::InterfacedBase::init() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#10 0x0000100846e34948 in std::_Mem_fn<void (ThePEG::InterfacedBase::*)()> std::for_each<std::_Rb_tree_const_iterator<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> >, std::_Mem_fn<void (ThePEG::InterfacedBase::*)()> >(std::_Rb_tree_const_iterator<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> >, std::_Rb_tree_const_iterator<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> >, std::_Mem_fn<void (ThePEG::InterfacedBase::*)()>) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#11 0x0000100846e156a4 in ThePEG::EventGenerator::doinit() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#12 0x0000100846e19a2c in ThePEG::EventGenerator::setup(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::set<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase>, std::less<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> >, std::allocator<ThePEG::Pointer::RCPtr<ThePEG::InterfacedBase> > >&, std::map<long, ThePEG::Pointer::RCPtr<ThePEG::ParticleData>, std::less<long>, std::allocator<std::pair<long const, ThePEG::Pointer::RCPtr<ThePEG::ParticleData> > > >&, std::set<ThePEG::Pointer::RCPtr<ThePEG::MatcherBase>, std::less<ThePEG::Pointer::RCPtr<ThePEG::MatcherBase> >, std::allocator<ThePEG::Pointer::RCPtr<ThePEG::MatcherBase> > >&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#13 0x0000100846e77518 in ThePEG::Repository::makeRun(ThePEG::Pointer::TransientRCPtr<ThePEG::EventGenerator>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#14 0x0000100846e7a4f0 in ThePEG::Repository::exec(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::ostream&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#15 0x0000100846e7b360 in ThePEG::Repository::execAndCheckReply(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::ostream&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#16 0x0000100846e7b6ec in ThePEG::Repository::read(std::istream&, std::ostream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#17 0x0000100846e7bca4 in ThePEG::Repository::read(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::ostream&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libThePEG.so.30
#18 0x000010084578a354 in (anonymous namespace)::HerwigGenericRead(Herwig::HerwigUI const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/external/el8_ppc64le_gcc12/lib/libHerwigAPI.so.2
#19 0x0000100845013f58 in Herwig7Interface::callHerwigGenerator() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/libGeneratorInterfaceHerwig7Interface.so
#20 0x0000100845016f94 in Herwig7Interface::initRepository(edm::ParameterSet const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/libGeneratorInterfaceHerwig7Interface.so
#21 0x0000100840f6af3c in Herwig7Hadronizer::Herwig7Hadronizer(edm::ParameterSet const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginGeneratorInterfaceHerwig7HadronizerPlugins.so
#22 0x0000100840f75dd8 in edm::GeneratorFilter<Herwig7Hadronizer, gen::ExternalDecayDriver>::GeneratorFilter(edm::ParameterSet const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginGeneratorInterfaceHerwig7HadronizerPlugins.so
#23 0x0000100840f761d8 in edm::WorkerMaker<edm::GeneratorFilter<Herwig7Hadronizer, gen::ExternalDecayDriver> >::makeModule(edm::ParameterSet const&) const () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginGeneratorInterfaceHerwig7HadronizerPlugins.so
#24 0x00001000004015dc in edm::Maker::makeModule(edm::MakeModuleParams const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#25 0x00001000002da9b0 in edm::Factory::makeModule(edm::MakeModuleParams const&, edm::ModuleTypeResolverMaker const*, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#26 0x00001000002f9048 in edm::ModuleRegistry::getModule(edm::MakeModuleParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#27 0x0000100000405a14 in edm::WorkerRegistry::getWorker(edm::WorkerParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#28 0x00001000004025c8 in edm::WorkerManager::getWorker(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#29 0x00001000003bf45c in edm::(anonymous namespace)::getWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::ParameterSet&, edm::WorkerManager&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#30 0x00001000003c4c18 in edm::StreamSchedule::fillWorkers(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::vector<edm::WorkerInPath, std::allocator<edm::WorkerInPath> >&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#31 0x00001000003c594c in edm::StreamSchedule::fillTrigPath(edm::ParameterSet&, edm::ProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<edm::HLTGlobalStatus>, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#32 0x00001000003c8790 in edm::StreamSchedule::StreamSchedule(std::shared_ptr<edm::TriggerResultInserter>, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::StreamID, edm::ProcessContext const*) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#33 0x00001000003a7358 in void std::_Construct<edm::StreamSchedule, std::shared_ptr<edm::TriggerResultInserter>&, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>&, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>&, std::shared_ptr<edm::ProcessConfiguration const>&, edm::StreamID, edm::ProcessContext const*&>(edm::StreamSchedule*, std::shared_ptr<edm::TriggerResultInserter>&, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>&, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>&, std::shared_ptr<edm::ProcessConfiguration const>&, edm::StreamID&&, edm::ProcessContext const*&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#34 0x0000100000392160 in edm::Schedule::Schedule(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::ProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::PreallocationConfiguration const&, edm::ProcessContext const*, edm::ModuleTypeResolverMaker const*) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#35 0x00001000003a9ad4 in edm::ScheduleItems::initModules(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProcessContext const*, edm::ModuleTypeResolverMaker const*) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#36 0x00001000002337c0 in tbb::detail::d1::function_task<edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy)::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#37 0x00001000025bac44 in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter> (t=0x10000382f500, waiter=..., this=0x100003823e00) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:322
#38 tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::outermost_worker_waiter> (t=0x0, waiter=..., this=0x100003823e00) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458
#39 tbb::detail::r1::arena::process (tls=..., this=0x100003823780) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/arena.cpp:137
#40 tbb::detail::r1::market::process (this=0x10000384ab00, j=...) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/market.cpp:599
#41 0x00001000025bde30 in tbb::detail::r1::rml::private_worker::run (this=0x100008c10100) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:271
#42 tbb::detail::r1::rml::private_worker::thread_routine (arg=0x100008c10100) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:221
#43 0x0000100002ca9718 in start_thread () from /lib64/libpthread.so.0
#44 0x0000100002e2b778 in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x10083fb291f0 (LWP 27891) "cmsRun"):
#0  0x0000100002de5b04 in nanosleep () from /lib64/libc.so.6
#1  0x0000100002de5940 in sleep () from /lib64/libc.so.6
#2  0x0000100009ed595c in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  <signal handler called>
#4  0x0000100002e22ac4 in syscall () from /lib64/libc.so.6
#5  0x00001000025be308 in tbb::detail::r1::futex_wait (comparand=2, futex=0x100008c100a4) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:100
#6  tbb::detail::r1::binary_semaphore::P (this=0x100008c100a4) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:253
#7  tbb::detail::r1::rml::internal::thread_monitor::wait (this=0x100008c100a0) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/rml_thread_monitor.h:235
#8  tbb::detail::r1::rml::private_worker::run (this=0x100008c10080) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:273
#9  tbb::detail::r1::rml::private_worker::thread_routine (arg=0x100008c10080) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/private_server.cpp:221
#10 0x0000100002ca9718 in start_thread () from /lib64/libpthread.so.0
#11 0x0000100002e2b778 in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x10082c7691f0 (LWP 27811) "cmsRun"):
#0  0x0000100002cb9510 in waitpid () from /lib64/libpthread.so.0
#1  0x0000100009ed5c3c in edm::service::cmssw_stacktrace_fork() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#2  0x0000100009ed7a5c in edm::service::InitRootHandlers::stacktraceHelperThread() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  0x0000100009edc924 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run() () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#4  0x0000100002964fa0 in std::execute_native_thread_routine (__p=<optimized out>) at ../../../../../libstdc++-v3/src/c++11/thread.cc:82
#5  0x0000100002ca9718 in start_thread () from /lib64/libpthread.so.0
#6  0x0000100002e2b778 in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x10000279d530 (LWP 26578) "cmsRun"):
#0  0x0000100002de5b04 in nanosleep () from /lib64/libc.so.6
#1  0x0000100002de5940 in sleep () from /lib64/libc.so.6
#2  0x0000100009ed595c in sig_pause_for_stacktrace () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw/CMSSW_14_1_X_2024-02-18-2300/lib/el8_ppc64le_gcc12/pluginFWCoreServicesPlugins.so
#3  <signal handler called>
#4  0x0000100002e22ac4 in syscall () from /lib64/libc.so.6
#5  0x00001000025c6148 in tbb::detail::r1::futex_wait (comparand=2, futex=0x3fffeea9b2f0) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:100
#6  tbb::detail::r1::binary_semaphore::P (this=0x3fffeea9b2f0) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/semaphore.h:253
#7  tbb::detail::r1::sleep_node<tbb::detail::r1::market_context>::wait (this=0x3fffeea9b2c0) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/concurrent_monitor.h:170
#8  tbb::detail::r1::concurrent_monitor_base<tbb::detail::r1::market_context>::commit_wait (this=<optimized out>, node=...) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/concurrent_monitor.h:232
#9  tbb::detail::r1::concurrent_monitor_base<tbb::detail::r1::market_context>::commit_wait (node=..., this=<optimized out>) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/concurrent_monitor.h:228
#10 tbb::detail::r1::concurrent_monitor_base<tbb::detail::r1::market_context>::wait<tbb::detail::r1::sleep_node<tbb::detail::r1::market_context>, tbb::detail::r1::external_waiter::pause(tbb::detail::r1::arena_slot&)::{lambda()#1}&>(tbb::detail::r1::external_waiter::pause(tbb::detail::r1::arena_slot&)::{lambda()#1}&, tbb::detail::r1::sleep_node<tbb::detail::r1::market_context>&&) (node=..., pred=<synthetic pointer>..., this=0x10000384ab18) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/concurrent_monitor.h:262
#11 tbb::detail::r1::sleep_waiter::sleep<tbb::detail::r1::external_waiter::pause(tbb::detail::r1::arena_slot&)::{lambda()#1}>(unsigned long, tbb::detail::r1::external_waiter::pause(tbb::detail::r1::arena_slot&)::{lambda()#1}) (this=0x3fffeea9b448, wakeup_condition=..., uniq_tag=<optimized out>) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/waiters.h:118
#12 tbb::detail::r1::external_waiter::pause (this=0x3fffeea9b448) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/waiters.h:144
#13 tbb::detail::r1::external_waiter::pause (this=0x3fffeea9b448) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/waiters.h:137
#14 tbb::detail::r1::task_dispatcher::receive_or_steal_task<true, tbb::detail::r1::external_waiter> (this=0x100003823d80, tls=..., ed=..., waiter=..., isolation=<optimized out>, fifo_allowed=<optimized out>, critical_allowed=<optimized out>) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:231
#15 0x00001000025c7d80 in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter> (waiter=..., t=0x0, this=0x100003823d80) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:350
#16 tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter> (waiter=..., t=<optimized out>, this=0x100003823d80) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458
#17 tbb::detail::r1::task_dispatcher::execute_and_wait (t=<optimized out>, wait_ctx=..., w_ctx=...) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/task_dispatcher.cpp:168
#18 0x000010000023cb30 in edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#19 0x000010000023fe80 in edm::EventProcessor::EventProcessor(std::shared_ptr<edm::ProcessDesc>, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) () from /cvmfs/cms-ib.cern.ch/sw/ppc64le/nweek-02825/el8_ppc64le_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-02-20-2300/lib/el8_ppc64le_gcc12/libFWCoreFramework.so
#20 0x0000000010007838 in tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>::operator()() const ()
#21 0x00001000025a9d60 in tbb::detail::r1::task_arena_impl::execute (ta=..., d=...) at /scratch/cmsbuild/jenkins_b/workspace/auto-builds/CMSSW_14_0_0_pre3-el8_ppc64le_gcc12/build/CMSSW_14_0_0_pre3-build/BUILD/el8_ppc64le_gcc12/external/tbb/v2021.9.0-16196f107c456d12eb9d248792d8cb7b/tbb-v2021.9.0/src/tbb/arena.cpp:688
#22 0x0000000010008dec in main::{lambda()#1}::operator()() const ()
#23 0x0000000010005b74 in main ()

Current Modules:

Module: none (crashed)

A fatal system signal has occurred: segmentation violation

The RelVal itself was introduced in #43692 and it is consistently failing for both 14_1 and 14_0 release cycles.

Thanks!
Andrea

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions