Open
Description
What did you do to encounter the bug?
Steps to reproduce the behavior:
- Brought up mongodb community edition cluster on k8s cluster (1 control plane and 2 worker nodes) on baremetal servers.
- Mongodb has 3 replicas spread on 2 worker nodes.
- We are using longhorn storage provider for dynamic PV creation.
- Did a warm reboot of the one of the worker node.
What did you expect?
After worker node reboot , i expected all 3 mongodb replicas to recover successfully.
What happened instead?
One of the replica remained in crashloopback state. It didnot recover itself.
Screenshots
NA
Operator Information
- Operator Version : v0.11.0
- MongoDB Image used :
- mongod container : mongodb-community-server:8.0.0-ubi8
- mongodb-agent container: mongodb-agent-ubi:107.0.7.8596-1
Kubernetes Cluster Information
- Distribution : Vanilla K8s ..
- Version : v1.29.5
- Image Registry location (quay, or an internal registry)
Additional context
NA
If possible, please include:
- The operator logs
- Below we assume that your replicaset database pods are named
mongo-<>. For instance:
root@node1:/home/labuser# kubectl get pods -n mongodb
NAME READY STATUS RESTARTS AGE
mongodb-0 2/2 Running 0 6m14s
mongodb-1 0/2 CrashLoopBackOff 5 (118s ago) 5m31s
mongodb-kubernetes-operator-6d7f45687c-96x9s 1/1 Running 1 (37m ago) 7h8m
root@node1:/home/labuser# kubectl exec -it -n mongodb mongodb-1 -c mongodb-agent -- cat /var/log/mongodb-mms-automation/healthstatus/agent-health-status.json
{"statuses":{"mongodb-1":{"IsInGoalState":false,"LastMongoUpTime":0,"ExpectedToBeUp":true,"ReplicationStatus":-1}},"mmsStatus":{"mongodb-1":{"name":"mongodb-1","lastGoalVersionAchieved":-1,"plans":[{"automationConfigVersion":1,"started":"2025-05-13T12:22:08.634594384Z","completed":null,"moves":[{"move":"Start","moveDoc":"Start the process","steps":[{"step":"StartFresh","stepDoc":"Start a mongo instance (start fresh)","isWaitStep":false,"started":"2025-05-13T12:22:08.634613181Z","completed":null,"result":""}]},{"move":"WaitRsInit","moveDoc":"Wait for the replica set to be initialized by another member","steps":[{"step":"WaitRsInit","stepDoc":"Wait for the replica set to be initialized by another member","isWaitStep":true,"started":null,"completed":null,"result":""}]},{"move":"WaitFeatureCompatibilityVersionCorrect","moveDoc":"Wait for featureCompatibilityVersion to be right","steps":[{"step":"WaitFeatureCompatibilityVersionCorrect","stepDoc":"Wait for featureCompatibilityVersion to be right","isWaitStep":true,"started":null,"completed":null,"result":""}]}]}],"errorCode":0,"errorString":"","waitDetails":{"RunSetParameter":"process not up","UpdateFeatureCompatibilityVersion":"process isn't up","WaitAllRsMembersUp":"[mongodb-1.mongodb-svc.mongodb.svc.cluster.local:27017: not healthy mongodb-2.mongodb-svc.mongodb.svc.cluster.local:27017: not healthy]","WaitCannotBecomePrimary":"Wait until the process is reconfigured with priority=0 by a different process","WaitClusterReadyForFCVUpdate":"process isn't up","WaitDefaultRWConcernCorrect":"waiting for the primary to update defaultRWConcern","WaitForResyncPrimaryManualInterventionStep":"A resync was requested on a primary. This requires manual intervention","WaitHealthyMajority":"[mongodb-1.mongodb-svc.mongodb.svc.cluster.local:27017: not healthy mongodb-2.mongodb-svc.mongodb.svc.cluster.local:27017: not healthy]","WaitMultipleHealthyNonArbiters":"[mongodb-1.mongodb-svc.mongodb.svc.cluster.local:27017: not healthy mongodb-2.mongodb-svc.mongodb.svc.cluster.local:27017: not healthy]","WaitNecessaryRsMembersUpForReconfig":"[mongodb-1.mongodb-svc.mongodb.svc.cluster.local:27017: not healthy mongodb-2.mongodb-svc.mongodb.svc.cluster.local:27017: not healthy]","WaitPrimary":"This process is expected to be the primary member. Check that the replica set state allows a primary to be elected","WaitProcessUp":"The process is running, but not yet responding to agent calls","WaitResetPlacementHistory":"config servers haven't seen the marker"}}}}
root@node1:/home/labuser# kubectl describe pod -n mongodb mongodb-1
Warning Unhealthy 5m53s kubelet Readiness probe failed: {"level":"info","ts":1747138934.2361245,"msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{state:0 sema:0} millCh:<nil> startMill:{done:{_:{} v:0} m:{state:0 sema:0}}}"}
{"level":"error","ts":1747138934.2387526,"msg":"health status file not avaible yet: open /var/log/mongodb-mms-automation/healthstatus/agent-health-status.json: no such file or directory "}
Warning Unhealthy 5m53s kubelet Readiness probe failed: {"level":"info","ts":1747138934.4183202,"msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{state:0 sema:0} millCh:<nil> startMill:{done:{_:{} v:0} m:{state:0 sema:0}}}"}
{"level":"error","ts":1747138934.4184098,"msg":"health status file not avaible yet: open /var/log/mongodb-mms-automation/healthstatus/agent-health-status.json: no such file or directory "}
Warning Unhealthy 5m52s kubelet Readiness probe failed: {"level":"info","ts":1747138935.4322023,"msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{state:0 sema:0} millCh:<nil> startMill:{done:{_:{} v:0} m:{state:0 sema:0}}}"}
{"level":"error","ts":1747138935.4323251,"msg":"health status file not avaible yet: open /var/log/mongodb-mms-automation/healthstatus/agent-health-status.json: no such file or directory "}
Warning Unhealthy 5m49s kubelet Readiness probe failed: {"level":"info","ts":1747138938.4527547,"msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{state:0 sema:0} millCh:<nil> startMill:{done:{_:{} v:0} m:{state:0 sema:0}}}"}
{"level":"info","ts":1747138938.4881268,"msg":"Mongod is not ready"}
{"level":"info","ts":1747138938.4881718,"msg":"Reached the end of the check. Returning not ready."}
Warning Unhealthy 5m48s kubelet Readiness probe failed: {"level":"info","ts":1747138939.4610236,"msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{state:0 sema:0} millCh:<nil> startMill:{done:{_:{} v:0} m:{state:0 sema:0}}}"}
{"level":"info","ts":1747138939.4959278,"msg":"Mongod is not ready"}
{"level":"info","ts":1747138939.4960153,"msg":"Reached the end of the check. Returning not ready."}
Warning BackOff 58s (x23 over 5m49s) kubelet Back-off restarting failed container mongod in pod mongodb-1_mongodb(9cba9e5e-35e7-4a74-b85b-712672c09816)
root@node1:/home/labuser#
root@node1:/home/labuser# kubectl logs -n mongodb mongodb-1
Defaulted container "mongod" out of: mongod, mongodb-agent, mongod-posthook (init), mongodb-agent-readinessprobe (init)
2025-05-13T12:28:19.160Z INFO versionhook/main.go:33 Running version change post-start hook
2025-05-13T12:28:19.160Z INFO versionhook/main.go:40 Waiting for agent health status...
2025-05-13T12:28:20.161Z INFO versionhook/main.go:76 Pod should not be deleted, mongod started
{"t":{"$date":"2025-05-13T12:28:22.207Z"},"s":"I", "c":"CONTROL", "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"authenticationMechanisms":{"default":["MONGODB-X509","SCRAM-SHA-1","SCRAM-SHA-256"],"value":["SCRAM-SHA-256"]}}}}
{"t":{"$date":"2025-05-13T12:28:22.226+00:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2025-05-13T12:28:22.228+00:00"},"s":"I", "c":"CONTROL", "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
{"t":{"$date":"2025-05-13T12:28:22.228+00:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set at least one of the related parameters","attr":{"relatedParameters":["tcpFastOpenServer","tcpFastOpenClient","tcpFastOpenQueueSize"]}}
{"t":{"$date":"2025-05-13T12:28:22.250+00:00"},"s":"I", "c":"NETWORK", "id":4915701, "ctx":"main","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":25},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":25},"outgoing":{"minWireVersion":6,"maxWireVersion":25},"isInternalClient":true}}}
{"t":{"$date":"2025-05-13T12:28:22.250+00:00"},"s":"I", "c":"TENANT_M", "id":7091600, "ctx":"main","msg":"Starting TenantMigrationAccessBlockerRegistry"}
{"t":{"$date":"2025-05-13T12:28:22.250+00:00"},"s":"I", "c":"CONTROL", "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data","architecture":"64-bit","host":"mongodb-1"}}
{"t":{"$date":"2025-05-13T12:28:22.250+00:00"},"s":"W", "c":"CONTROL", "id":20720, "ctx":"initandlisten","msg":"Memory available to mongo process is less than total system memory","attr":{"availableMemSizeMB":2048,"systemMemSizeMB":515464}}
{"t":{"$date":"2025-05-13T12:28:22.250+00:00"},"s":"I", "c":"CONTROL", "id":23403, "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"8.0.0","gitVersion":"d7cd03b239ac39a3c7d63f7145e91aca36f93db6","openSSLVersion":"OpenSSL 1.1.1k FIPS 25 Mar 2021","modules":[],"allocator":"tcmalloc-google","environment":{"distmod":"rhel88","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2025-05-13T12:28:22.250+00:00"},"s":"I", "c":"CONTROL", "id":51765, "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Red Hat Enterprise Linux release 8.10 (Ootpa)","version":"Kernel 6.8.0-59-generic"}}}
{"t":{"$date":"2025-05-13T12:28:22.250+00:00"},"s":"I", "c":"CONTROL", "id":21951, "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/data/automation-mongod.conf","net":{"bindIp":"0.0.0.0","port":27017},"replication":{"replSetName":"mongodb"},"security":{"authorization":"enabled","keyFile":"/var/lib/mongodb-mms-automation/authentication/keyfile"},"setParameter":{"authenticationMechanisms":"SCRAM-SHA-256"},"storage":{"dbPath":"/data","engine":"wiredTiger","wiredTiger":{"engineConfig":{"cacheSizeGB":0.5,"journalCompressor":"zlib"}}}}}}
{"t":{"$date":"2025-05-13T12:28:22.251+00:00"},"s":"W", "c":"STORAGE", "id":22271, "ctx":"initandlisten","msg":"Detected unclean shutdown - Lock file is not empty","attr":{"lockFile":"/data/mongod.lock"}}
{"t":{"$date":"2025-05-13T12:28:22.251+00:00"},"s":"W", "c":"STORAGE", "id":22302, "ctx":"initandlisten","msg":"Recovering data from the last clean checkpoint."}
{"t":{"$date":"2025-05-13T12:28:22.251+00:00"},"s":"I", "c":"STORAGE", "id":22297, "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
{"t":{"$date":"2025-05-13T12:28:22.251+00:00"},"s":"I", "c":"STORAGE", "id":22315, "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=512M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=zlib),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],prefetch=(available=true,default=false),"}}
{"t":{"$date":"2025-05-13T12:28:22.502+00:00"},"s":"E", "c":"WT", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":-31809,"message":{"ts_sec":1747139302,"ts_usec":502021,"thread":"1:0x79e6c5e0c100","session_name":"connection","category":"WT_VERB_DEFAULT","category_id":12,"verbose_level":"ERROR","verbose_level_id":-3,"msg":"__wti_turtle_read:672:WiredTiger.turtle: fatal turtle file read error","error_str":"WT_TRY_SALVAGE: database corruption detected","error_code":-31809}}}
{"t":{"$date":"2025-05-13T12:28:22.502+00:00"},"s":"E", "c":"WT", "id":22435, "ctx":"initandlisten","msg":"WiredTiger error message","attr":{"error":-31804,"message":{"ts_sec":1747139302,"ts_usec":502124,"thread":"1:0x79e6c5e0c100","session_name":"connection","category":"WT_VERB_DEFAULT","category_id":12,"verbose_level":"ERROR","verbose_level_id":-3,"msg":"__wti_turtle_read:672:the process must exit and restart","error_str":"WT_PANIC: WiredTiger library panic","error_code":-31804}}}
{"t":{"$date":"2025-05-13T12:28:22.502+00:00"},"s":"F", "c":"ASSERT", "id":23089, "ctx":"initandlisten","msg":"Fatal assertion","attr":{"msgid":50853,"file":"src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp","line":791}}
{"t":{"$date":"2025-05-13T12:28:22.502+00:00"},"s":"F", "c":"ASSERT", "id":23090, "ctx":"initandlisten","msg":"\n\n***aborting after fassert() failure\n\n"}
{"t":{"$date":"2025-05-13T12:28:22.502+00:00"},"s":"F", "c":"CONTROL", "id":6384300, "ctx":"initandlisten","msg":"Writing fatal message","attr":{"message":"\n"}}
{"t":{"$date":"2025-05-13T12:28:22.502+00:00"},"s":"F", "c":"CONTROL", "id":6384300, "ctx":"initandlisten","msg":"Writing fatal message","attr":{"message":"Got signal: 6 (Aborted).\n"}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31380, "ctx":"initandlisten","msg":"BACKTRACE","attr":{"bt":{"backtrace":[{"a":"65276EF0FE18","b":"6527660EF000","o":"8E20E18","s":"_ZN5mongo15printStackTraceEv","C":"mongo::printStackTrace()","s+":"38"},{"a":"65276EF0A9F7","b":"6527660EF000","o":"8E1B9F7","s":"abruptQuit","s+":"77"},{"a":"79E6C4415990","b":"79E6C4403000","o":"12990","s":"funlockfile","s+":"50"},{"a":"79E6C407B52F","b":"79E6C402D000","o":"4E52F","s":"gsignal","s+":"10F"},{"a":"79E6C404EE65","b":"79E6C402D000","o":"21E65","s":"abort","s+":"127"},{"a":"65276EEFB791","b":"6527660EF000","o":"8E0C791","s":"_ZN5mongo12_GLOBAL__N_19callAbortEv","C":"mongo::(anonymous namespace)::callAbort()","s+":"1B"},{"a":"65276EEFC9D3","b":"6527660EF000","o":"8E0D9D3","s":"_ZN5mongo14fassert_detail6failedENS_14SourceLocationENS0_5MsgIdE","C":"mongo::fassert_detail::failed(mongo::SourceLocation, mongo::fassert_detail::MsgId)","s+":"165"},{"a":"65276BE6C3BF","b":"6527660EF000","o":"5D7D3BF","s":"_ZN5mongo12_GLOBAL__N_141mdb_handle_error_with_startup_suppressionEP18__wt_event_handlerP12__wt_sessioniPKc.cold","C":"mongo::(anonymous namespace)::mdb_handle_error_with_startup_suppression(__wt_event_handler*, __wt_session*, int, char const*) [clone .cold]","s+":"16"},{"a":"65276C0483A7","b":"6527660EF000","o":"5F593A7","s":"__eventv","s+":"10D7"},{"a":"65276C048FC3","b":"6527660EF000","o":"5F59FC3","s":"__wt_panic_func","s+":"154"},{"a":"65276BF9570B","b":"6527660EF000","o":"5EA670B","s":"__wti_turtle_read.cold","s+":"58"},{"a":"65276BF93EC9","b":"6527660EF000","o":"5EA4EC9","s":"__wt_turtle_validate_version","s+":"179"},{"a":"65276BE84637","b":"6527660EF000","o":"5D95637","s":"wiredtiger_open","s+":"2C87"},{"a":"65276BE18E41","b":"6527660EF000","o":"5D29E41","s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_","C":"mongo::WiredTigerKVEngine::_openWiredTiger(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)","s+":"A1"},{"a":"65276BE2189B","b":"6527660EF000","o":"5D3289B","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbb","C":"mongo::WiredTigerKVEngine::WiredTigerKVEngine(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::ClockSource*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, unsigned long, bool, bool)","s+":"B8B"},{"a":"65276BDE1374","b":"6527660EF000","o":"5CF2374","s":"_ZNK5mongo12_GLOBAL__N_117WiredTigerFactory6createEPNS_16OperationContextERKNS_19StorageGlobalParamsEPKNS_21StorageEngineLockFileE","C":"mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::OperationContext*, mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const*) const","s+":"154"},{"a":"65276C189141","b":"6527660EF000","o":"609A141","s":"_ZN5mongo23initializeStorageEngineEPNS_16OperationContextENS_22StorageEngineInitFlagsEPNS_14BSONObjBuilderE","C":"mongo::initializeStorageEngine(mongo::OperationContext*, mongo::StorageEngineInitFlags, mongo::BSONObjBuilder*)","s+":"441"},{"a":"65276A75E1AD","b":"6527660EF000","o":"466F1AD","s":"_ZN5mongo12_GLOBAL__N_114_initAndListenEPNS_14ServiceContextEi.constprop.0","C":"mongo::(anonymous namespace)::_initAndListen(mongo::ServiceContext*, int) [clone .constprop.0]","s+":"6FD"},{"a":"65276A762232","b":"6527660EF000","o":"4673232","s":"_ZN5mongo11mongod_mainEiPPc","C":"mongo::mongod_main(int, char**)","s+":"952"},{"a":"65276A757549","b":"6527660EF000","o":"4668549","s":"main","s+":"9"},{"a":"79E6C40677E5","b":"79E6C402D000","o":"3A7E5","s":"__libc_start_main","s+":"E5"},{"a":"65276A75742E","b":"6527660EF000","o":"466842E","s":"_start","s+":"2E"}],"processInfo":{"mongodbVersion":"8.0.0","gitVersion":"d7cd03b239ac39a3c7d63f7145e91aca36f93db6","compiledModules":[],"uname":{"sysname":"Linux","release":"6.8.0-59-generic","version":"#61~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 17:03:15 UTC 2","machine":"x86_64"},"somap":[{"b":"6527660EF000","elfType":3,"buildId":"C1DAB2FBBBD2123D"},{"b":"79E6C4403000","path":"/lib64/libpthread.so.0","elfType":3,"buildId":"FD0441CF253F643CA977DDFBC46257EDDAF544AE"},{"b":"79E6C402D000","path":"/lib64/libc.so.6","elfType":3,"buildId":"889235A2805B8308B2D0274921BBE1890E9A1986"}]}}},"tags":[]}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276EF0FE18","b":"6527660EF000","o":"8E20E18","s":"_ZN5mongo15printStackTraceEv","C":"mongo::printStackTrace()","s+":"38"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276EF0A9F7","b":"6527660EF000","o":"8E1B9F7","s":"abruptQuit","s+":"77"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"79E6C4415990","b":"79E6C4403000","o":"12990","s":"funlockfile","s+":"50"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"79E6C407B52F","b":"79E6C402D000","o":"4E52F","s":"gsignal","s+":"10F"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"79E6C404EE65","b":"79E6C402D000","o":"21E65","s":"abort","s+":"127"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276EEFB791","b":"6527660EF000","o":"8E0C791","s":"_ZN5mongo12_GLOBAL__N_19callAbortEv","C":"mongo::(anonymous namespace)::callAbort()","s+":"1B"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276EEFC9D3","b":"6527660EF000","o":"8E0D9D3","s":"_ZN5mongo14fassert_detail6failedENS_14SourceLocationENS0_5MsgIdE","C":"mongo::fassert_detail::failed(mongo::SourceLocation, mongo::fassert_detail::MsgId)","s+":"165"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BE6C3BF","b":"6527660EF000","o":"5D7D3BF","s":"_ZN5mongo12_GLOBAL__N_141mdb_handle_error_with_startup_suppressionEP18__wt_event_handlerP12__wt_sessioniPKc.cold","C":"mongo::(anonymous namespace)::mdb_handle_error_with_startup_suppression(__wt_event_handler*, __wt_session*, int, char const*) [clone .cold]","s+":"16"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276C0483A7","b":"6527660EF000","o":"5F593A7","s":"__eventv","s+":"10D7"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276C048FC3","b":"6527660EF000","o":"5F59FC3","s":"__wt_panic_func","s+":"154"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BF9570B","b":"6527660EF000","o":"5EA670B","s":"__wti_turtle_read.cold","s+":"58"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BF93EC9","b":"6527660EF000","o":"5EA4EC9","s":"__wt_turtle_validate_version","s+":"179"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BE84637","b":"6527660EF000","o":"5D95637","s":"wiredtiger_open","s+":"2C87"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BE18E41","b":"6527660EF000","o":"5D29E41","s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_","C":"mongo::WiredTigerKVEngine::_openWiredTiger(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)","s+":"A1"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BE2189B","b":"6527660EF000","o":"5D3289B","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbb","C":"mongo::WiredTigerKVEngine::WiredTigerKVEngine(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::ClockSource*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, unsigned long, bool, bool)","s+":"B8B"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BDE1374","b":"6527660EF000","o":"5CF2374","s":"_ZNK5mongo12_GLOBAL__N_117WiredTigerFactory6createEPNS_16OperationContextERKNS_19StorageGlobalParamsEPKNS_21StorageEngineLockFileE","C":"mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::OperationContext*, mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const*) const","s+":"154"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276C189141","b":"6527660EF000","o":"609A141","s":"_ZN5mongo23initializeStorageEngineEPNS_16OperationContextENS_22StorageEngineInitFlagsEPNS_14BSONObjBuilderE","C":"mongo::initializeStorageEngine(mongo::OperationContext*, mongo::StorageEngineInitFlags, mongo::BSONObjBuilder*)","s+":"441"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276A75E1AD","b":"6527660EF000","o":"466F1AD","s":"_ZN5mongo12_GLOBAL__N_114_initAndListenEPNS_14ServiceContextEi.constprop.0","C":"mongo::(anonymous namespace)::_initAndListen(mongo::ServiceContext*, int) [clone .constprop.0]","s+":"6FD"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276A762232","b":"6527660EF000","o":"4673232","s":"_ZN5mongo11mongod_mainEiPPc","C":"mongo::mongod_main(int, char**)","s+":"952"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276A757549","b":"6527660EF000","o":"4668549","s":"main","s+":"9"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"79E6C40677E5","b":"79E6C402D000","o":"3A7E5","s":"__libc_start_main","s+":"E5"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276A75742E","b":"6527660EF000","o":"466842E","s":"_start","s+":"2E"}}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"F", "c":"CONTROL", "id":6384300, "ctx":"initandlisten","msg":"Writing fatal message","attr":{"message":"Invalid access at address: 0\n"}}
{"t":{"$date":"2025-05-13T12:28:22.563+00:00"},"s":"F", "c":"CONTROL", "id":6384300, "ctx":"initandlisten","msg":"Writing fatal message","attr":{"message":"Dumping siginfo (si_code=128): 0b 00 00 00 00 00 00 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00\n"}}
{"t":{"$date":"2025-05-13T12:28:22.564+00:00"},"s":"F", "c":"CONTROL", "id":6384300, "ctx":"initandlisten","msg":"Writing fatal message","attr":{"message":"Got signal: 11 (Segmentation fault).\n"}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31380, "ctx":"initandlisten","msg":"BACKTRACE","attr":{"bt":{"backtrace":[{"a":"65276EF0FE18","b":"6527660EF000","o":"8E20E18","s":"_ZN5mongo15printStackTraceEv","C":"mongo::printStackTrace()","s+":"38"},{"a":"65276EF0A8F9","b":"6527660EF000","o":"8E1B8F9","s":"abruptQuitWithAddrSignal","s+":"139"},{"a":"79E6C4415990","b":"79E6C4403000","o":"12990","s":"funlockfile","s+":"50"},{"a":"79E6C404EF41","b":"79E6C402D000","o":"21F41","s":"abort","s+":"203"},{"a":"65276EEFB791","b":"6527660EF000","o":"8E0C791","s":"_ZN5mongo12_GLOBAL__N_19callAbortEv","C":"mongo::(anonymous namespace)::callAbort()","s+":"1B"},{"a":"65276EEFC9D3","b":"6527660EF000","o":"8E0D9D3","s":"_ZN5mongo14fassert_detail6failedENS_14SourceLocationENS0_5MsgIdE","C":"mongo::fassert_detail::failed(mongo::SourceLocation, mongo::fassert_detail::MsgId)","s+":"165"},{"a":"65276BE6C3BF","b":"6527660EF000","o":"5D7D3BF","s":"_ZN5mongo12_GLOBAL__N_141mdb_handle_error_with_startup_suppressionEP18__wt_event_handlerP12__wt_sessioniPKc.cold","C":"mongo::(anonymous namespace)::mdb_handle_error_with_startup_suppression(__wt_event_handler*, __wt_session*, int, char const*) [clone .cold]","s+":"16"},{"a":"65276C0483A7","b":"6527660EF000","o":"5F593A7","s":"__eventv","s+":"10D7"},{"a":"65276C048FC3","b":"6527660EF000","o":"5F59FC3","s":"__wt_panic_func","s+":"154"},{"a":"65276BF9570B","b":"6527660EF000","o":"5EA670B","s":"__wti_turtle_read.cold","s+":"58"},{"a":"65276BF93EC9","b":"6527660EF000","o":"5EA4EC9","s":"__wt_turtle_validate_version","s+":"179"},{"a":"65276BE84637","b":"6527660EF000","o":"5D95637","s":"wiredtiger_open","s+":"2C87"},{"a":"65276BE18E41","b":"6527660EF000","o":"5D29E41","s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_","C":"mongo::WiredTigerKVEngine::_openWiredTiger(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)","s+":"A1"},{"a":"65276BE2189B","b":"6527660EF000","o":"5D3289B","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbb","C":"mongo::WiredTigerKVEngine::WiredTigerKVEngine(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::ClockSource*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, unsigned long, bool, bool)","s+":"B8B"},{"a":"65276BDE1374","b":"6527660EF000","o":"5CF2374","s":"_ZNK5mongo12_GLOBAL__N_117WiredTigerFactory6createEPNS_16OperationContextERKNS_19StorageGlobalParamsEPKNS_21StorageEngineLockFileE","C":"mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::OperationContext*, mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const*) const","s+":"154"},{"a":"65276C189141","b":"6527660EF000","o":"609A141","s":"_ZN5mongo23initializeStorageEngineEPNS_16OperationContextENS_22StorageEngineInitFlagsEPNS_14BSONObjBuilderE","C":"mongo::initializeStorageEngine(mongo::OperationContext*, mongo::StorageEngineInitFlags, mongo::BSONObjBuilder*)","s+":"441"},{"a":"65276A75E1AD","b":"6527660EF000","o":"466F1AD","s":"_ZN5mongo12_GLOBAL__N_114_initAndListenEPNS_14ServiceContextEi.constprop.0","C":"mongo::(anonymous namespace)::_initAndListen(mongo::ServiceContext*, int) [clone .constprop.0]","s+":"6FD"},{"a":"65276A762232","b":"6527660EF000","o":"4673232","s":"_ZN5mongo11mongod_mainEiPPc","C":"mongo::mongod_main(int, char**)","s+":"952"},{"a":"65276A757549","b":"6527660EF000","o":"4668549","s":"main","s+":"9"},{"a":"79E6C40677E5","b":"79E6C402D000","o":"3A7E5","s":"__libc_start_main","s+":"E5"},{"a":"65276A75742E","b":"6527660EF000","o":"466842E","s":"_start","s+":"2E"}],"processInfo":{"mongodbVersion":"8.0.0","gitVersion":"d7cd03b239ac39a3c7d63f7145e91aca36f93db6","compiledModules":[],"uname":{"sysname":"Linux","release":"6.8.0-59-generic","version":"#61~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue Apr 15 17:03:15 UTC 2","machine":"x86_64"},"somap":[{"b":"6527660EF000","elfType":3,"buildId":"C1DAB2FBBBD2123D"},{"b":"79E6C4403000","path":"/lib64/libpthread.so.0","elfType":3,"buildId":"FD0441CF253F643CA977DDFBC46257EDDAF544AE"},{"b":"79E6C402D000","path":"/lib64/libc.so.6","elfType":3,"buildId":"889235A2805B8308B2D0274921BBE1890E9A1986"}]}}},"tags":[]}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276EF0FE18","b":"6527660EF000","o":"8E20E18","s":"_ZN5mongo15printStackTraceEv","C":"mongo::printStackTrace()","s+":"38"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276EF0A8F9","b":"6527660EF000","o":"8E1B8F9","s":"abruptQuitWithAddrSignal","s+":"139"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"79E6C4415990","b":"79E6C4403000","o":"12990","s":"funlockfile","s+":"50"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"79E6C404EF41","b":"79E6C402D000","o":"21F41","s":"abort","s+":"203"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276EEFB791","b":"6527660EF000","o":"8E0C791","s":"_ZN5mongo12_GLOBAL__N_19callAbortEv","C":"mongo::(anonymous namespace)::callAbort()","s+":"1B"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276EEFC9D3","b":"6527660EF000","o":"8E0D9D3","s":"_ZN5mongo14fassert_detail6failedENS_14SourceLocationENS0_5MsgIdE","C":"mongo::fassert_detail::failed(mongo::SourceLocation, mongo::fassert_detail::MsgId)","s+":"165"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BE6C3BF","b":"6527660EF000","o":"5D7D3BF","s":"_ZN5mongo12_GLOBAL__N_141mdb_handle_error_with_startup_suppressionEP18__wt_event_handlerP12__wt_sessioniPKc.cold","C":"mongo::(anonymous namespace)::mdb_handle_error_with_startup_suppression(__wt_event_handler*, __wt_session*, int, char const*) [clone .cold]","s+":"16"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276C0483A7","b":"6527660EF000","o":"5F593A7","s":"__eventv","s+":"10D7"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276C048FC3","b":"6527660EF000","o":"5F59FC3","s":"__wt_panic_func","s+":"154"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BF9570B","b":"6527660EF000","o":"5EA670B","s":"__wti_turtle_read.cold","s+":"58"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BF93EC9","b":"6527660EF000","o":"5EA4EC9","s":"__wt_turtle_validate_version","s+":"179"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BE84637","b":"6527660EF000","o":"5D95637","s":"wiredtiger_open","s+":"2C87"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BE18E41","b":"6527660EF000","o":"5D29E41","s":"_ZN5mongo18WiredTigerKVEngine15_openWiredTigerERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_","C":"mongo::WiredTigerKVEngine::_openWiredTiger(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)","s+":"A1"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BE2189B","b":"6527660EF000","o":"5D3289B","s":"_ZN5mongo18WiredTigerKVEngineC2ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES8_PNS_11ClockSourceES8_mmbb","C":"mongo::WiredTigerKVEngine::WiredTigerKVEngine(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, mongo::ClockSource*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, unsigned long, bool, bool)","s+":"B8B"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276BDE1374","b":"6527660EF000","o":"5CF2374","s":"_ZNK5mongo12_GLOBAL__N_117WiredTigerFactory6createEPNS_16OperationContextERKNS_19StorageGlobalParamsEPKNS_21StorageEngineLockFileE","C":"mongo::(anonymous namespace)::WiredTigerFactory::create(mongo::OperationContext*, mongo::StorageGlobalParams const&, mongo::StorageEngineLockFile const*) const","s+":"154"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276C189141","b":"6527660EF000","o":"609A141","s":"_ZN5mongo23initializeStorageEngineEPNS_16OperationContextENS_22StorageEngineInitFlagsEPNS_14BSONObjBuilderE","C":"mongo::initializeStorageEngine(mongo::OperationContext*, mongo::StorageEngineInitFlags, mongo::BSONObjBuilder*)","s+":"441"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276A75E1AD","b":"6527660EF000","o":"466F1AD","s":"_ZN5mongo12_GLOBAL__N_114_initAndListenEPNS_14ServiceContextEi.constprop.0","C":"mongo::(anonymous namespace)::_initAndListen(mongo::ServiceContext*, int) [clone .constprop.0]","s+":"6FD"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276A762232","b":"6527660EF000","o":"4673232","s":"_ZN5mongo11mongod_mainEiPPc","C":"mongo::mongod_main(int, char**)","s+":"952"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276A757549","b":"6527660EF000","o":"4668549","s":"main","s+":"9"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"79E6C40677E5","b":"79E6C402D000","o":"3A7E5","s":"__libc_start_main","s+":"E5"}}}
{"t":{"$date":"2025-05-13T12:28:22.622+00:00"},"s":"I", "c":"CONTROL", "id":31445, "ctx":"initandlisten","msg":"Frame","attr":{"frame":{"a":"65276A75742E","b":"6527660EF000","o":"466842E","s":"_start","s+":"2E"}}}
Metadata
Metadata
Assignees
Labels
No labels