Skip to content

automation-mongod.conf says "bindIp: 0.0.0.0", which does not work on ipv6 cluster #1191

Description

@Infinoid

What did you do to encounter the bug?
Steps to reproduce the behavior:

  1. Deploy mongodb-kubernetes to ipv6-enabled EKS cluster via helm chart
  2. try to provision a simple database using MongoDBCommunity CR
  3. repeatedly kubectl describe the pod
  4. see that it never becomes Ready

What did you expect?
Pod becomes ready, mongodb up and running.

What happened instead?
Pod never becomes Ready, mongodb-agent cannot connect to mongod.

Operator Information

  • Operator Version: 1.8.1 (installed via helm chart)
  • MongoDB Image used: 6.0.5-ubi8 (also tried 8.3.2-ubi8, no difference)

Kubernetes Cluster Information

  • Distribution: Amazon EKS
  • Version: v1.35.4-eks-40737a8
  • Image Registry location: quay.io/mongodb/mongodb-community-server

Additional context

  • mongod is configured (in /data/automation-mongod.conf) to bind to address "0.0.0.0".
  • mongodb-agent appears to be connecting to the pod IP, which is an IPv6 address.
  • mongodb-agent fails to connect to mongod, because mongod is only listening on IPv4.
  • As a result, the mongodb-agent container never enters the Ready state.

kubernetes state:

% kc -n mongodb get pod
NAME                                          READY   STATUS    RESTARTS   AGE
mongodb-kubernetes-operator-ffd67dd9c-qj859   1/1     Running   0          13m
simple-0                                      1/2     Running   0          4m13s

% kc -n mongodb get mdbc
NAME     PHASE     VERSION
simple   Pending

MongoDBCommunity manifest (cobbled together from public/samples/community/mongodb.com_v1_mongodbcommunity_cr.yaml and mongodb-community-operator/controllers/testdata/custom_storage_class.yaml):

apiVersion: mongodbcommunity.mongodb.com/v1
kind: MongoDBCommunity
metadata:
  name: simple
spec:
  members: 1
  type: ReplicaSet
  version: "6.0.5"
  security:
    authentication:
      modes: ["SCRAM"]
  users:
    - name: my-user
      db: admin
      passwordSecretRef: # a reference to the secret that will be used to generate the user's password
        name: my-user-password
      roles:
        - name: clusterAdmin
          db: admin
        - name: userAdminAnyDatabase
          db: admin
      scramCredentialsSecretName: my-scram
  statefulSet:
    spec:
      volumeClaimTemplates:
        - metadata:
            name: data-volume
          spec:
            accessModes: [ "ReadWriteOnce" ]
            storageClassName: "encrypted-gp3"
            resources:
              requests:
                storage: 1Gi
        - metadata:
            name: logs-volume
          spec:
            accessModes: [ "ReadWriteOnce" ]
            storageClassName: "encrypted-gp3"
            resources:
              requests:
                storage: 1Gi

---
apiVersion: v1
kind: Secret
metadata:
  name: my-user-password
type: Opaque
stringData:
  password: abc123

Generated mongod config file:

% kc -n mongodb exec pod/simple-0 --container=mongod -- cat /data/automation-mongod.conf
net:
  bindIp: 0.0.0.0
  port: 27017
replication:
  replSetName: simple
security:
  authorization: enabled
  keyFile: /var/lib/mongodb-mms-automation/authentication/keyfile
setParameter:
  authenticationMechanisms: SCRAM-SHA-256
storage:
  dbPath: /data

Output of kubectl describe pod, note that the pod IP is an IPv6 address:

% kc -n mongodb describe pod/simple-0
Name:             simple-0
Namespace:        mongodb
Priority:         0
Service Account:  mongodb-kubernetes-appdb
Node:             i-0c1f8fa545351d3ba.ec2.internal/2600:1f18:7bbe:ba49::2e15
Start Time:       Wed, 03 Jun 2026 11:38:19 -0400
Labels:           app=simple-svc
                  apps.kubernetes.io/pod-index=0
                  controller-revision-hash=simple-77845f7d98
                  statefulset.kubernetes.io/pod-name=simple-0
                  topology.kubernetes.io/region=us-east-1
                  topology.kubernetes.io/zone=us-east-1c
Annotations:      agent.mongodb.com/version: -1
Status:           Running
IP:               2600:1f18:7bbe:ba49:2d15::13
IPs:
  IP:           2600:1f18:7bbe:ba49:2d15::13
Controlled By:  StatefulSet/simple
Init Containers:
  mongod-posthook:
    Container ID:  containerd://e3305ff3177d07046f32343ff091652d5a9dcdb0b54921c5def0269b4b2528c0
    Image:         quay.io/mongodb/mongodb-kubernetes-operator-version-upgrade-post-start-hook:1.0.10
    Image ID:      quay.io/mongodb/mongodb-kubernetes-operator-version-upgrade-post-start-hook@sha256:f321ec1d25d6e98805b8be9321f2a926d702835136dde88d5fffe917c2df1d0a
    Port:          <none>
    Host Port:     <none>
    Command:
      cp
      version-upgrade-hook
      /hooks/version-upgrade
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Wed, 03 Jun 2026 11:38:25 -0400
      Finished:     Wed, 03 Jun 2026 11:38:25 -0400
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  500M
    Requests:
      cpu:        500m
      memory:     400M
    Environment:  <none>
    Mounts:
      /hooks from hooks (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tngjd (ro)
  mongodb-agent-readinessprobe:
    Container ID:  containerd://3d94b6a925688a22ed5eacd4f4d601ce99676e1f9906033c22b3ab20090b452b
    Image:         quay.io/mongodb/mongodb-kubernetes-readinessprobe:1.0.24
    Image ID:      quay.io/mongodb/mongodb-kubernetes-readinessprobe@sha256:eb325a42fee5d90cf87d0643d4c70fb49fe06ef9724494ba055de521dcc8140a
    Port:          <none>
    Host Port:     <none>
    Command:
      cp
      /probes/readinessprobe
      /opt/scripts/readinessprobe
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Wed, 03 Jun 2026 11:38:25 -0400
      Finished:     Wed, 03 Jun 2026 11:38:25 -0400
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  500M
    Requests:
      cpu:        500m
      memory:     400M
    Environment:  <none>
    Mounts:
      /opt/scripts from agent-scripts (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tngjd (ro)
Containers:
  mongod:
    Container ID:  containerd://f8bac5e7800f91f6339aa6ee3baf8890031f8395f6d50eea81148a485a825d4f
    Image:         quay.io/mongodb/mongodb-community-server:6.0.5-ubi8
    Image ID:      quay.io/mongodb/mongodb-community-server@sha256:18f5bfd202ecb50d4f1b2625af6042b0a0f7d2c05408000671ac8f1cc6d300dd
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -c

      if [ -e "/hooks/version-upgrade" ]; then
        #run post-start hook to handle version changes (if exists)
          /hooks/version-upgrade
      fi

      # wait for config and keyfile to be created by the agent
      echo "Waiting for config and keyfile files to be created by the agent..."
      while ! [ -f /data/automation-mongod.conf -a -f /var/lib/mongodb-mms-automation/authentication/keyfile ]; do
        sleep 3;
        echo "Waiting..."
      done

      # sleep is important after agent issues shutdown command
      # k8s restarts the mongod container too quickly for the agent to realize mongod is down
      echo "Sleeping for 15s..."
      sleep 15

      # start mongod with this configuration
      echo "Starting mongod..."
      exec mongod -f /data/automation-mongod.conf

    Args:

    State:          Running
      Started:      Wed, 03 Jun 2026 11:38:26 -0400
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  500M
    Requests:
      cpu:     500m
      memory:  400M
    Environment:
      AGENT_STATUS_FILEPATH:  /healthstatus/agent-health-status.json
    Mounts:
      /data from data-volume (rw)
      /healthstatus from healthstatus (rw)
      /hooks from hooks (rw)
      /tmp from tmp (rw)
      /var/lib/mongodb-mms-automation/authentication from simple-keyfile (rw)
      /var/log/mongodb-mms-automation from logs-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tngjd (ro)
  mongodb-agent:
    Container ID:  containerd://30e4792f6a12048a4c60d68bbe198923941c2d5fcb997f633014cdbbd30b4ed7
    Image:         quay.io/mongodb/mongodb-agent:108.0.2.8729-1
    Image ID:      quay.io/mongodb/mongodb-agent@sha256:b422829aac63f068aed4a6948fd14cbd3c3bfad6d47ab76523f6839bf20b0203
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/bash
      -c
      current_uid=$(id -u)
      declare -r current_uid
      if ! grep -q "${current_uid}" /etc/passwd ; then
      sed -e "s/^mongodb:/builder:/" /etc/passwd > /tmp/passwd
      echo "mongodb:x:$(id -u):$(id -g):,,,:/:/bin/bash" >> /tmp/passwd
      export NSS_WRAPPER_PASSWD=/tmp/passwd
      export LD_PRELOAD=libnss_wrapper.so
      export NSS_WRAPPER_GROUP=/etc/group
      fi
      agent/mongodb-agent -healthCheckFilePath=/var/log/mongodb-mms-automation/healthstatus/agent-health-status.json -serveStatusPort=5000 -cluster=/var/lib/automation/config/cluster-config.json -skipMongoStart -noDaemonize -useLocalMongoDbTools -logFile /var/log/mongodb-mms-automation/automation-agent.log -logLevel INFO -maxLogFileDurationHrs 24
    State:          Running
      Started:      Wed, 03 Jun 2026 11:38:26 -0400
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  500M
    Requests:
      cpu:      500m
      memory:   400M
    Readiness:  exec [/opt/scripts/readinessprobe] delay=5s timeout=1s period=10s #success=1 #failure=40
    Environment:
      AGENT_STATUS_FILEPATH:  /var/log/mongodb-mms-automation/healthstatus/agent-health-status.json
      AUTOMATION_CONFIG_MAP:  simple-config
      HEADLESS_AGENT:         true
      POD_NAMESPACE:          mongodb (v1:metadata.namespace)
    Mounts:
      /data from data-volume (rw)
      /opt/scripts from agent-scripts (rw)
      /tmp from tmp (rw)
      /var/lib/automation/config from automation-config (ro)
      /var/lib/mongodb-mms-automation/authentication from simple-keyfile (rw)
      /var/log/mongodb-mms-automation from logs-volume (rw)
      /var/log/mongodb-mms-automation/healthstatus from healthstatus (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tngjd (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True
  Initialized                 True
  Ready                       False
  ContainersReady             False
  PodScheduled                True
Volumes:
  data-volume:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  data-volume-simple-0
    ReadOnly:   false
  logs-volume:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  logs-volume-simple-0
    ReadOnly:   false
  agent-scripts:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  automation-config:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  simple-config
    Optional:    false
  healthstatus:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  hooks:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  simple-keyfile:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  tmp:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  kube-api-access-tngjd:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    Optional:                false
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason                  Age   From                     Message
  ----     ------                  ----  ----                     -------
  Normal   Scheduled               107s  default-scheduler        Successfully assigned mongodb/simple-0 to i-0c1f8fa545351d3ba.ec2.internal
  Normal   SuccessfulAttachVolume  105s  attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-c938e411-62a5-4475-8cf5-fdf43e7e935f"
  Normal   SuccessfulAttachVolume  104s  attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-16bbae19-10a9-4bea-87ee-94a322ec4a54"
  Normal   Pulling                 101s  kubelet                  spec.initContainers{mongod-posthook}: Pulling image "quay.io/mongodb/mongodb-kubernetes-operator-version-upgrade-post-start-hook:1.0.10"
  Normal   Pulled                  101s  kubelet                  spec.initContainers{mongod-posthook}: Successfully pulled image "quay.io/mongodb/mongodb-kubernetes-operator-version-upgrade-post-start-hook:1.0.10" in 109ms (109ms including waiting). Image size: 56095648 bytes.
  Normal   Created                 101s  kubelet                  spec.initContainers{mongod-posthook}: Container created
  Normal   Started                 101s  kubelet                  spec.initContainers{mongod-posthook}: Container started
  Normal   Pulling                 101s  kubelet                  spec.initContainers{mongodb-agent-readinessprobe}: Pulling image "quay.io/mongodb/mongodb-kubernetes-readinessprobe:1.0.24"
  Normal   Pulled                  101s  kubelet                  spec.initContainers{mongodb-agent-readinessprobe}: Successfully pulled image "quay.io/mongodb/mongodb-kubernetes-readinessprobe:1.0.24" in 95ms (95ms including waiting). Image size: 61701623 bytes.
  Normal   Created                 101s  kubelet                  spec.initContainers{mongodb-agent-readinessprobe}: Container created
  Normal   Started                 101s  kubelet                  spec.initContainers{mongodb-agent-readinessprobe}: Container started
  Normal   Pulled                  100s  kubelet                  spec.containers{mongod}: Container image "quay.io/mongodb/mongodb-community-server:6.0.5-ubi8" already present on machine and can be accessed by the pod
  Normal   Created                 100s  kubelet                  spec.containers{mongod}: Container created
  Normal   Started                 100s  kubelet                  spec.containers{mongod}: Container started
  Normal   Pulling                 100s  kubelet                  spec.containers{mongodb-agent}: Pulling image "quay.io/mongodb/mongodb-agent:108.0.2.8729-1"
  Normal   Pulled                  100s  kubelet                  spec.containers{mongodb-agent}: Successfully pulled image "quay.io/mongodb/mongodb-agent:108.0.2.8729-1" in 128ms (128ms including waiting). Image size: 174333100 bytes.
  Normal   Created                 100s  kubelet                  spec.containers{mongodb-agent}: Container created
  Normal   Started                 100s  kubelet                  spec.containers{mongodb-agent}: Container started
  Warning  Unhealthy               88s   kubelet                  spec.containers{mongodb-agent}: Readiness probe failed: {"level":"info","ts":"2026-06-03T15:38:38.538Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:38:38.583Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:38:38.583Z","msg":"Reached the end of the check. Returning not ready."}
  Warning  Unhealthy  88s  kubelet  spec.containers{mongodb-agent}: Readiness probe failed: {"level":"info","ts":"2026-06-03T15:38:38.664Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:38:38.706Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:38:38.706Z","msg":"Reached the end of the check. Returning not ready."}
  Warning  Unhealthy  78s  kubelet  spec.containers{mongodb-agent}: Readiness probe failed: {"level":"info","ts":"2026-06-03T15:38:48.675Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:38:48.717Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:38:48.717Z","msg":"Reached the end of the check. Returning not ready."}
  Warning  Unhealthy  68s  kubelet  spec.containers{mongodb-agent}: Readiness probe failed: {"level":"info","ts":"2026-06-03T15:38:58.652Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:38:58.691Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:38:58.691Z","msg":"Reached the end of the check. Returning not ready."}
  Warning  Unhealthy  58s  kubelet  spec.containers{mongodb-agent}: Readiness probe failed: {"level":"info","ts":"2026-06-03T15:39:08.664Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:39:08.742Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:39:08.743Z","msg":"Reached the end of the check. Returning not ready."}
  Warning  Unhealthy  48s  kubelet  spec.containers{mongodb-agent}: Readiness probe failed: {"level":"info","ts":"2026-06-03T15:39:18.663Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:39:18.712Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:39:18.713Z","msg":"Reached the end of the check. Returning not ready."}
  Warning  Unhealthy  38s  kubelet  spec.containers{mongodb-agent}: Readiness probe failed: {"level":"info","ts":"2026-06-03T15:39:28.643Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:39:28.684Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:39:28.684Z","msg":"Reached the end of the check. Returning not ready."}
  Warning  Unhealthy  28s  kubelet  spec.containers{mongodb-agent}: Readiness probe failed: {"level":"info","ts":"2026-06-03T15:39:38.663Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:39:38.718Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:39:38.718Z","msg":"Reached the end of the check. Returning not ready."}
  Warning  Unhealthy  22s  kubelet  spec.containers{mongodb-agent}: Readiness probe failed: {"level":"info","ts":"2026-06-03T15:39:44.232Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:39:44.307Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:39:44.307Z","msg":"Reached the end of the check. Returning not ready."}
  Warning  Unhealthy  2s (x2 over 12s)  kubelet  spec.containers{mongodb-agent}: (combined from similar events): Readiness probe failed: {"level":"info","ts":"2026-06-03T15:40:04.215Z","msg":"logging configuration: &{Filename:/var/log/mongodb-mms-automation/readiness.log MaxSize:5 MaxAge:0 MaxBackups:5 LocalTime:false Compress:false size:0 file:<nil> mu:{_:{} mu:{state:0 sema:0}} millCh:<nil> startMill:{_:{} done:{_:{} v:0} m:{_:{} mu:{state:0 sema:0}}}}"}
{"level":"info","ts":"2026-06-03T15:40:04.252Z","msg":"Mongod is not ready"}
{"level":"info","ts":"2026-06-03T15:40:04.252Z","msg":"Reached the end of the check. Returning not ready."}

mongod logs, note that it binds to 0.0.0.0:

2026-06-03T15:38:26.620Z        INFO    versionhook/main.go:33  Running version change post-start hook
2026-06-03T15:38:26.620Z        INFO    versionhook/main.go:40  Waiting for agent health status...
2026-06-03T15:38:27.621Z        INFO    versionhook/main.go:46  Agent health status file not found, mongod will start
Waiting for config and keyfile files to be created by the agent...
Sleeping for 15s...
Starting mongod...
{"t":{"$date":"2026-06-03T15:38:42.650Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"-","msg":"Applied --setParameter options","attr":{"serverParameters":{"authenticationMechanisms":{"default":["MONGODB-X509","SCRAM-SHA-1","SCRAM-SHA-256"],"value":["SCRAM-SHA-256"]}}}}
{"t":{"$date":"2026-06-03T15:38:42.653+00:00"},"s":"I",  "c":"NETWORK",  "id":4915701, "ctx":"-","msg":"Initialized wire specification","attr":{"spec":{"incomingExternalClient":{"minWireVersion":0,"maxWireVersion":17},"incomingInternalClient":{"minWireVersion":0,"maxWireVersion":17},"outgoing":{"minWireVersion":6,"maxWireVersion":17},"isInternalClient":true}}}
{"t":{"$date":"2026-06-03T15:38:42.653+00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2026-06-03T15:38:42.654+00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2026-06-03T15:38:42.680+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationDonorService","namespace":"config.tenantMigrationDonors"}}
{"t":{"$date":"2026-06-03T15:38:42.680+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"TenantMigrationRecipientService","namespace":"config.tenantMigrationRecipients"}}
{"t":{"$date":"2026-06-03T15:38:42.680+00:00"},"s":"I",  "c":"REPL",     "id":5123008, "ctx":"main","msg":"Successfully registered PrimaryOnlyService","attr":{"service":"ShardSplitDonorService","namespace":"config.tenantSplitDonors"}}
{"t":{"$date":"2026-06-03T15:38:42.680+00:00"},"s":"I",  "c":"CONTROL",  "id":5945603, "ctx":"main","msg":"Multi threading initialized"}
{"t":{"$date":"2026-06-03T15:38:42.680+00:00"},"s":"I",  "c":"CONTROL",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data","architecture":"64-bit","host":"simple-0"}}
{"t":{"$date":"2026-06-03T15:38:42.680+00:00"},"s":"W",  "c":"CONTROL",  "id":20720,   "ctx":"initandlisten","msg":"Memory available to mongo process is less than total system memory","attr":{"availableMemSizeMB":476,"systemMemSizeMB":7809}}
{"t":{"$date":"2026-06-03T15:38:42.680+00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"6.0.5","gitVersion":"c9a99c120371d4d4c52cbb15dac34a36ce8d3b1d","openSSLVersion":"OpenSSL 1.1.1k  FIPS 25 Mar 2021","modules":[],"allocator":"tcmalloc","environment":{"distmod":"rhel82","distarch":"aarch64","target_arch":"aarch64"}}}}
{"t":{"$date":"2026-06-03T15:38:42.680+00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Red Hat Enterprise Linux release 8.10 (Ootpa)","version":"Kernel 6.12.80-106.156.amzn2023.aarch64"}}}
{"t":{"$date":"2026-06-03T15:38:42.680+00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"config":"/data/automation-mongod.conf","net":{"bindIp":"0.0.0.0","port":27017},"replication":{"replSetName":"simple"},"security":{"authorization":"enabled","keyFile":"/var/lib/mongodb-mms-automation/authentication/keyfile"},"setParameter":{"authenticationMechanisms":"SCRAM-SHA-256"},"storage":{"dbPath":"/data"}}}}
{"t":{"$date":"2026-06-03T15:38:42.681+00:00"},"s":"I",  "c":"STORAGE",  "id":22297,   "ctx":"initandlisten","msg":"Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem","tags":["startupWarnings"]}
{"t":{"$date":"2026-06-03T15:38:42.681+00:00"},"s":"I",  "c":"STORAGE",  "id":22315,   "ctx":"initandlisten","msg":"Opening WiredTiger","attr":{"config":"create,cache_size=256M,session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,remove=true,path=journal,compressor=snappy),builtin_extension_config=(zstd=(compression_level=6)),file_manager=(close_idle_time=600,close_scan_interval=10,close_handle_minimum=2000),statistics_log=(wait=0),json_output=(error,message),verbose=[recovery_progress:1,checkpoint_progress:1,compact_progress:1,backup:0,checkpoint:0,compact:0,evict:0,history_store:0,recovery:0,rts:0,salvage:0,tiered:0,timestamp:0,transaction:0,verify:0,log:0],"}}
{"t":{"$date":"2026-06-03T15:38:43.448+00:00"},"s":"I",  "c":"STORAGE",  "id":4795906, "ctx":"initandlisten","msg":"WiredTiger opened","attr":{"durationMillis":767}}
{"t":{"$date":"2026-06-03T15:38:43.448+00:00"},"s":"I",  "c":"RECOVERY", "id":23987,   "ctx":"initandlisten","msg":"WiredTiger recoveryTimestamp","attr":{"recoveryTimestamp":{"$timestamp":{"t":0,"i":0}}}}
{"t":{"$date":"2026-06-03T15:38:43.466+00:00"},"s":"I",  "c":"REPL",     "id":5853300, "ctx":"initandlisten","msg":"current featureCompatibilityVersion value","attr":{"featureCompatibilityVersion":"unset","context":"startup"}}
{"t":{"$date":"2026-06-03T15:38:43.466+00:00"},"s":"I",  "c":"STORAGE",  "id":5071100, "ctx":"initandlisten","msg":"Clearing temp directory"}
{"t":{"$date":"2026-06-03T15:38:43.466+00:00"},"s":"I",  "c":"CONTROL",  "id":20536,   "ctx":"initandlisten","msg":"Flow Control is enabled on this deployment"}
{"t":{"$date":"2026-06-03T15:38:43.466+00:00"},"s":"I",  "c":"FTDC",     "id":20625,   "ctx":"initandlisten","msg":"Initializing full-time diagnostic data capture","attr":{"dataDirectory":"/data/diagnostic.data"}}
{"t":{"$date":"2026-06-03T15:38:43.471+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"local.startup_log","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"705bc00d-a609-4d79-b058-31e5aa984a2a"}},"options":{"capped":true,"size":10485760}}}
{"t":{"$date":"2026-06-03T15:38:43.483+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"705bc00d-a609-4d79-b058-31e5aa984a2a"}},"namespace":"local.startup_log","index":"_id_","ident":"index-1-7459984585964580897","collectionIdent":"collection-0-7459984585964580897","commitTimestamp":null}}
{"t":{"$date":"2026-06-03T15:38:43.484+00:00"},"s":"I",  "c":"REPL",     "id":6015317, "ctx":"initandlisten","msg":"Setting new configuration state","attr":{"newState":"ConfigStartingUp","oldState":"ConfigPreStart"}}
{"t":{"$date":"2026-06-03T15:38:43.484+00:00"},"s":"I",  "c":"REPL",     "id":6005300, "ctx":"initandlisten","msg":"Starting up replica set aware services"}
{"t":{"$date":"2026-06-03T15:38:43.484+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"ReadConcernMajorityNotAvailableYet: Read concern majority reads are currently not possible.","nextWakeupMillis":200}}
{"t":{"$date":"2026-06-03T15:38:43.485+00:00"},"s":"I",  "c":"REPL",     "id":4280500, "ctx":"initandlisten","msg":"Attempting to create internal replication collections"}
{"t":{"$date":"2026-06-03T15:38:43.485+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"local.replset.oplogTruncateAfterPoint","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"8f1ab210-9cd8-42f3-9edd-385f76f87e94"}},"options":{}}}
{"t":{"$date":"2026-06-03T15:38:43.497+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"8f1ab210-9cd8-42f3-9edd-385f76f87e94"}},"namespace":"local.replset.oplogTruncateAfterPoint","index":"_id_","ident":"index-3-7459984585964580897","collectionIdent":"collection-2-7459984585964580897","commitTimestamp":null}}
{"t":{"$date":"2026-06-03T15:38:43.497+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"local.replset.minvalid","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"f81269c5-0e73-4b4f-a1c9-612d07cdbfef"}},"options":{}}}
{"t":{"$date":"2026-06-03T15:38:43.508+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"f81269c5-0e73-4b4f-a1c9-612d07cdbfef"}},"namespace":"local.replset.minvalid","index":"_id_","ident":"index-5-7459984585964580897","collectionIdent":"collection-4-7459984585964580897","commitTimestamp":null}}
{"t":{"$date":"2026-06-03T15:38:43.509+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"local.replset.election","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"42317c28-9a9a-44aa-9d70-9169212ffe53"}},"options":{}}}
{"t":{"$date":"2026-06-03T15:38:43.519+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"42317c28-9a9a-44aa-9d70-9169212ffe53"}},"namespace":"local.replset.election","index":"_id_","ident":"index-7-7459984585964580897","collectionIdent":"collection-6-7459984585964580897","commitTimestamp":null}}
{"t":{"$date":"2026-06-03T15:38:43.520+00:00"},"s":"I",  "c":"REPL",     "id":4280501, "ctx":"initandlisten","msg":"Attempting to load local voted for document"}
{"t":{"$date":"2026-06-03T15:38:43.520+00:00"},"s":"I",  "c":"REPL",     "id":21311,   "ctx":"initandlisten","msg":"Did not find local initialized voted for document at startup"}
{"t":{"$date":"2026-06-03T15:38:43.520+00:00"},"s":"I",  "c":"REPL",     "id":4280502, "ctx":"initandlisten","msg":"Searching for local Rollback ID document"}
{"t":{"$date":"2026-06-03T15:38:43.520+00:00"},"s":"I",  "c":"REPL",     "id":21312,   "ctx":"initandlisten","msg":"Did not find local Rollback ID document at startup. Creating one"}
{"t":{"$date":"2026-06-03T15:38:43.520+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"local.system.rollback.id","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"f2911a29-f173-4444-aedd-797442c632a2"}},"options":{}}}
{"t":{"$date":"2026-06-03T15:38:43.531+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"f2911a29-f173-4444-aedd-797442c632a2"}},"namespace":"local.system.rollback.id","index":"_id_","ident":"index-9-7459984585964580897","collectionIdent":"collection-8-7459984585964580897","commitTimestamp":null}}
{"t":{"$date":"2026-06-03T15:38:43.531+00:00"},"s":"I",  "c":"REPL",     "id":21531,   "ctx":"initandlisten","msg":"Initialized the rollback ID","attr":{"rbid":1}}
{"t":{"$date":"2026-06-03T15:38:43.531+00:00"},"s":"I",  "c":"REPL",     "id":21313,   "ctx":"initandlisten","msg":"Did not find local replica set configuration document at startup","attr":{"error":{"code":47,"codeName":"NoMatchingDocument","errmsg":"Did not find replica set configuration document in local.system.replset"}}}
{"t":{"$date":"2026-06-03T15:38:43.531+00:00"},"s":"I",  "c":"REPL",     "id":6015317, "ctx":"initandlisten","msg":"Setting new configuration state","attr":{"newState":"ConfigUninitialized","oldState":"ConfigStartingUp"}}
{"t":{"$date":"2026-06-03T15:38:43.532+00:00"},"s":"I",  "c":"STORAGE",  "id":20320,   "ctx":"initandlisten","msg":"createCollection","attr":{"namespace":"local.system.views","uuidDisposition":"generated","uuid":{"uuid":{"$uuid":"e72d2687-91f0-416a-985e-9060d7518723"}},"options":{}}}
{"t":{"$date":"2026-06-03T15:38:43.544+00:00"},"s":"I",  "c":"INDEX",    "id":20345,   "ctx":"initandlisten","msg":"Index build: done building","attr":{"buildUUID":null,"collectionUUID":{"uuid":{"$uuid":"e72d2687-91f0-416a-985e-9060d7518723"}},"namespace":"local.system.views","index":"_id_","ident":"index-11-7459984585964580897","collectionIdent":"collection-10-7459984585964580897","commitTimestamp":null}}
{"t":{"$date":"2026-06-03T15:38:43.545+00:00"},"s":"I",  "c":"STORAGE",  "id":22262,   "ctx":"initandlisten","msg":"Timestamp monitor starting"}
{"t":{"$date":"2026-06-03T15:38:43.546+00:00"},"s":"I",  "c":"CONTROL",  "id":20714,   "ctx":"LogicalSessionCacheRefresh","msg":"Failed to refresh session cache, will try again at the next refresh interval","attr":{"error":"NotYetInitialized: Replication has not yet been configured"}}
{"t":{"$date":"2026-06-03T15:38:43.547+00:00"},"s":"I",  "c":"CONTROL",  "id":20712,   "ctx":"LogicalSessionCacheReap","msg":"Sessions collection is not set up; waiting until next sessions reap interval","attr":{"error":"NamespaceNotFound: config.system.sessions does not exist"}}
{"t":{"$date":"2026-06-03T15:38:43.547+00:00"},"s":"I",  "c":"REPL",     "id":40440,   "ctx":"initandlisten","msg":"Starting the TopologyVersionObserver"}
{"t":{"$date":"2026-06-03T15:38:43.547+00:00"},"s":"I",  "c":"REPL",     "id":40445,   "ctx":"TopologyVersionObserver","msg":"Started TopologyVersionObserver"}
{"t":{"$date":"2026-06-03T15:38:43.548+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"/tmp/mongodb-27017.sock"}}
{"t":{"$date":"2026-06-03T15:38:43.548+00:00"},"s":"I",  "c":"NETWORK",  "id":23015,   "ctx":"listener","msg":"Listening on","attr":{"address":"0.0.0.0"}}
{"t":{"$date":"2026-06-03T15:38:43.548+00:00"},"s":"I",  "c":"NETWORK",  "id":23016,   "ctx":"listener","msg":"Waiting for connections","attr":{"port":27017,"ssl":"off"}}
{"t":{"$date":"2026-06-03T15:38:43.684+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":400}}
{"t":{"$date":"2026-06-03T15:38:44.085+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":600}}
{"t":{"$date":"2026-06-03T15:38:44.686+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":800}}
{"t":{"$date":"2026-06-03T15:38:45.487+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":1000}}
{"t":{"$date":"2026-06-03T15:38:46.488+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":1200}}
{"t":{"$date":"2026-06-03T15:38:47.689+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":1400}}
{"t":{"$date":"2026-06-03T15:38:49.091+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":1600}}
{"t":{"$date":"2026-06-03T15:38:50.693+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":1800}}
{"t":{"$date":"2026-06-03T15:38:52.494+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":2000}}
{"t":{"$date":"2026-06-03T15:38:54.496+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":2200}}
{"t":{"$date":"2026-06-03T15:38:56.699+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":2400}}
{"t":{"$date":"2026-06-03T15:38:59.101+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":2600}}
{"t":{"$date":"2026-06-03T15:39:01.704+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":2800}}
{"t":{"$date":"2026-06-03T15:39:04.505+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":3000}}
{"t":{"$date":"2026-06-03T15:39:07.509+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":3200}}
{"t":{"$date":"2026-06-03T15:39:10.712+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":3400}}
{"t":{"$date":"2026-06-03T15:39:14.114+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":3600}}
{"t":{"$date":"2026-06-03T15:39:17.718+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":3800}}
{"t":{"$date":"2026-06-03T15:39:21.519+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":4000}}
{"t":{"$date":"2026-06-03T15:39:25.521+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":4200}}
{"t":{"$date":"2026-06-03T15:39:29.726+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":4400}}
{"t":{"$date":"2026-06-03T15:39:34.129+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":4600}}
{"t":{"$date":"2026-06-03T15:39:38.734+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":4800}}
{"t":{"$date":"2026-06-03T15:39:43.539+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":5000}}
{"t":{"$date":"2026-06-03T15:39:48.544+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":5200}}
{"t":{"$date":"2026-06-03T15:39:53.749+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":5400}}
{"t":{"$date":"2026-06-03T15:39:59.151+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":5600}}
{"t":{"$date":"2026-06-03T15:40:04.756+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":5800}}
{"t":{"$date":"2026-06-03T15:40:10.560+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":6000}}
{"t":{"$date":"2026-06-03T15:40:16.561+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":6200}}
{"t":{"$date":"2026-06-03T15:40:22.768+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":6400}}
{"t":{"$date":"2026-06-03T15:40:29.175+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":6600}}
{"t":{"$date":"2026-06-03T15:40:35.779+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":6800}}
{"t":{"$date":"2026-06-03T15:40:42.586+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":7000}}
{"t":{"$date":"2026-06-03T15:40:49.589+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":7200}}
{"t":{"$date":"2026-06-03T15:40:56.790+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":7400}}
{"t":{"$date":"2026-06-03T15:41:04.196+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":7600}}
{"t":{"$date":"2026-06-03T15:41:11.799+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":7800}}
{"t":{"$date":"2026-06-03T15:41:19.604+00:00"},"s":"I",  "c":"-",        "id":4939300, "ctx":"monitoring-keys-for-HMAC","msg":"Failed to refresh key cache","attr":{"error":"NotYetInitialized: Cannot use non-local read concern until replica set is finished initializing.","nextWakeupMillis":8000}}

tail of mongodb-agent verbose logs, note that it attempts to connect to IPv6 address:

[2026-06-03T15:39:21.153+0000] [.info] [main/components/agent.go:LoadClusterConfig:276] [15:39:21.153] clusterConfig unchanged
[2026-06-03T15:39:21.173+0000] [.warn] [src/mongoclientservice/mongoclientservice.go:createClient:1288] [15:39:21.173] Server for client=0x0 connParams=simple-0.simple-svc.mongodb.svc.cluster.local:27017 (local=false,router=false) determined to be down due to error=[15:39:21.173] Non-TLS attempt failed : dial tcp [2600:1f18:7bbe:ba49:2d15::13]:27017: connect: connection refused
[2026-06-03T15:39:21.244+0000] [.warn] [src/mongoclientservice/mongoclientservice.go:createClient:1288] [15:39:21.244] Server for client=0x0 connParams=simple-0.simple-svc.mongodb.svc.cluster.local:27017 (local=false,router=false) determined to be down due to error=[15:39:21.244] Non-TLS attempt failed : dial tcp [2600:1f18:7bbe:ba49:2d15::13]:27017: connect: connection refused
[2026-06-03T15:39:21.244+0000] [.warn] [metrics/collector/util.go:getPingStatus:127] <hardwareMetricsCollector> [15:39:21.244] Failed to fetch replStatus for simple-0 : <hardwareMetricsCollector> [15:39:21.244] Server at simple-0.simple-svc.mongodb.svc.cluster.local:27017 (local=false,router=false) is down
[2026-06-03T15:39:21.306+0000] [.warn] [src/mongoclientservice/mongoclientservice.go:createClient:1288] [15:39:21.306] Server for client=0x0 connParams=simple-0.simple-svc.mongodb.svc.cluster.local:27017 (local=false,router=false) determined to be down due to error=[15:39:21.306] Non-TLS attempt failed : dial tcp [2600:1f18:7bbe:ba49:2d15::13]:27017: connect: connection refused
[2026-06-03T15:39:21.438+0000] [.warn] [src/mongoclientservice/mongoclientservice.go:createClient:1288] [15:39:21.438] Server for client=0x0 connParams=simple-0.simple-svc.mongodb.svc.cluster.local:27017 (local=false,router=false) determined to be down due to error=[15:39:21.438] Non-TLS attempt failed : dial tcp [2600:1f18:7bbe:ba49:2d15::13]:27017: connect: connection refused
[2026-06-03T15:39:21.570+0000] [.warn] [src/mongoclientservice/mongoclientservice.go:createClient:1288] [15:39:21.570] Server for client=0x0 connParams=simple-0.simple-svc.mongodb.svc.cluster.local:27017 (local=false,router=false) determined to be down due to error=[15:39:21.570] Non-TLS attempt failed : dial tcp [2600:1f18:7bbe:ba49:2d15::13]:27017: connect: connection refused
[2026-06-03T15:39:21.701+0000] [.warn] [src/mongoclientservice/mongoclientservice.go:createClient:1288] [15:39:21.701] Server for client=0x0 connParams=simple-0.simple-svc.mongodb.svc.cluster.local:27017 (local=false,router=false) determined to be down due to error=[15:39:21.701] Non-TLS attempt failed : dial tcp [2600:1f18:7bbe:ba49:2d15::13]:27017: connect: connection refused
[2026-06-03T15:39:21.833+0000] [.warn] [src/mongoclientservice/mongoclientservice.go:createClient:1288] [15:39:21.833] Server for client=0x0 connParams=simple-0.simple-svc.mongodb.svc.cluster.local:27017 (local=false,router=false) determined to be down due to error=[15:39:21.833] Non-TLS attempt failed : dial tcp [2600:1f18:7bbe:ba49:2d15::13]:27017: connect: connection refused
[2026-06-03T15:39:21.964+0000] [.warn] [src/mongoclientservice/mongoclientservice.go:createClient:1288] [15:39:21.964] Server for client=0x0 connParams=simple-0.simple-svc.mongodb.svc.cluster.local:27017 (local=false,router=false) determined to be down due to error=[15:39:21.964] Non-TLS attempt failed : dial tcp [2600:1f18:7bbe:ba49:2d15::13]:27017: connect: connection refused

mongodb-agent cluster-config.json:

% kc -n mongodb exec pod/simple-0 --container=mongodb-agent -- cat /var/lib/automation/config/cluster-config.json
{"version":1,"processes":[{"name":"simple-0","disabled":false,"hostname":"simple-0.simple-svc.mongodb.svc.cluster.local","args2_6":{"net":{"port":27017},"replication":{"replSetName":"simple"},"storage":{"dbPath":"/data"}},"featureCompatibilityVersion":"8.3","processType":"mongod","version":"8.3.2","authSchemaVersion":5}],"replicaSets":[{"_id":"simple","members":[{"_id":0,"host":"simple-0","arbiterOnly":false,"votes":1,"priority":1}],"protocolVersion":"1","numberArbiters":0}],"auth":{"usersWanted":[{"mechanisms":[],"roles":[{"role":"clusterAdmin","db":"admin"},{"role":"userAdminAnyDatabase","db":"admin"}],"user":"my-user","db":"admin","authenticationRestrictions":[],"scramSha256Creds":{"iterationCount":15000,"salt":"U0LHHLPVnmAu5U3fPNYLH48hUuD4q+FHEtzaDQ==","serverKey":"Ca2XQSlmuMBM95S+rRGFV31ZLma9OrEuHqkSzJTepcc=","storedKey":"Lw6A1k0bRjwGf8uxk+VhxMbqVtX4wfqCr9LCAsSK2Y4="},"scramSha1Creds":{"iterationCount":10000,"salt":"/pVeDR0YLJ65RiGFQmMUBw==","serverKey":"0Y2MR4S/zq45Mxh+HiAThzYpXBE=","storedKey":"+H331ZA1TrXm/Ca8Foc+692y1zk="}}],"disabled":false,"authoritativeSet":false,"autoAuthMechanisms":["SCRAM-SHA-256"],"autoAuthMechanism":"SCRAM-SHA-256","deploymentAuthMechanisms":["SCRAM-SHA-256"],"autoUser":"mms-automation","key":"Z5ru1fqUr8OUZZxpbTIsKC1l3K1oSX+ybsm15tmeoOdvssGBnvZaUqesVqsJb15p8wAjkTnEeiUxT/7My0I+E76RlFWYwIoxcPYD9kXehHhJftayFluVxWfLReh4aKaCB//02brhsEMdQEMOIkX/QE2PeF2dY99T4ISCiiEiMVQZbEA6OP4BlmIdyZzGDW20X70Q8FWciEawFw4IgqzESLJeQf5Q7x+QyQe7HG1kd+dsFOXqhB58gSxs1la3PhZ6oSnF7AJ/Bli4DUANTJlq85BtQVO9jdfKXrah6X+RoDM2gUpVsaxW8j54bO+xuJbA3bWnkO4/KazHLpuYQPsArEZ6MswsA7TlATXnqh4rIjzBNQ7Tsls6AUHyP8tSnjmHMSJEaW5R4vTiZhhSI6lH74Znc6jrc+Z5n1b4KrPB9fVeHZDbMKwZvrrrw3CNkonmVRYAn5x83JdKB1DSpIhVQcsgUe2kkdbMXnH5Q3REejwnmQrywEzkWgz2FK/Yji1nLTElU67gmbFLiZnxV9C7Xr+wZOUwkruStSXYSvE9ZNb33ne9LyBG2ssICB4iuvwo6GPJx34SFSkTtgUk5MawSmEcdGbfq6UHUkalXG7ha82uoAYLA5oze1B6m+3ObryGT1bCrwFFlGI5nlu+8T76ukf1+qk=","keyfile":"/var/lib/mongodb-mms-automation/authentication/keyfile","keyfileWindows":"%SystemDrive%\\MMSAutomation\\versions\\keyfile","autoPwd":"2AHZFb8LOeUPC1hU0r5x"},"tls":{"CAFilePath":"","clientCertificateMode":"OPTIONAL"},"mongoDbVersions":[{"name":"8.3.2","builds":[{"platform":"linux","url":"","gitVersion":"","architecture":"amd64","flavor":"rhel","minOsVersion":"","maxOsVersion":"","modules":[]},{"platform":"linux","url":"","gitVersion":"","architecture":"amd64","flavor":"ubuntu","minOsVersion":"","maxOsVersion":"","modules":[]},{"platform":"linux","url":"","gitVersion":"","architecture":"aarch64","flavor":"ubuntu","minOsVersion":"","maxOsVersion":"","modules":[]},{"platform":"linux","url":"","gitVersion":"","architecture":"aarch64","flavor":"rhel","minOsVersion":"","maxOsVersion":"","modules":[]}]}],"backupVersions":[],"monitoringVersions":[],"options":{"downloadBase":"/var/lib/mongodb-mms-automation"}}

Agent health status:

% kc -n mongodb exec pod/simple-0 --container=mongodb-agent -- cat /var/log/mongodb-mms-automation/healthstatus/agent-health-status.json
{"statuses":{"simple-0":{"IsInGoalState":false,"LastMongoUpTime":0,"ExpectedToBeUp":true,"ReplicationStatus":-1}},"mmsStatus":{"simple-0":{"name":"simple-0","lastGoalVersionAchieved":-1,"plans":[{"automationConfigVersion":1,"started":"2026-06-03T15:44:46.81993423Z","completed":null,"moves":[{"move":"Start","moveDoc":"Start the process","steps":[{"step":"StartFresh","stepDoc":"Start a mongo instance  (start fresh)","isWaitStep":false,"started":"2026-06-03T15:44:46.819964097Z","completed":null,"result":""}]},{"move":"WaitAllRsMembersUp","moveDoc":"Wait until all members of this process' repl set are up","steps":[{"step":"WaitAllRsMembersUp","stepDoc":"Wait until all members of this process' repl set are up","isWaitStep":true,"started":null,"completed":null,"result":""}]},{"move":"RsInit","moveDoc":"Initialize a replica set including the current MongoDB process","steps":[{"step":"RsInit","stepDoc":"Initialize a replica set","isWaitStep":false,"started":null,"completed":null,"result":""}]},{"move":"WaitFeatureCompatibilityVersionCorrect","moveDoc":"Wait for featureCompatibilityVersion to be right","steps":[{"step":"WaitFeatureCompatibilityVersionCorrect","stepDoc":"Wait for featureCompatibilityVersion to be right","isWaitStep":true,"started":null,"completed":null,"result":""}]}]}],"errorCode":0,"errorString":"","waitDetails":{"RunSetParameter":"process not up","UpdateFeatureCompatibilityVersion":"process isn't up","WaitAllRsMembersUp":"[]","WaitCannotBecomePrimary":"Wait until the process is reconfigured with priority=0 by a different process","WaitClusterReadyForFCVUpdate":"process isn't up","WaitDefaultRWConcernCorrect":"waiting for the primary to update defaultRWConcern","WaitForResyncPrimaryManualInterventionStep":"A resync was requested on a primary. This requires manual intervention","WaitHealthyMajority":"[]","WaitMultipleHealthyNonArbiters":"[]","WaitNecessaryRsMembersUpForReconfig":"[]","WaitPrimary":"This process is expected to be the primary member. Check that the replica set state allows a primary to be elected","WaitProcessUp":"The process is running, but not yet responding to agent calls","WaitResetPlacementHistory":"config servers  haven't seen the marker"}}}}

I believe the mongod should bind to "::" instead. This would accept connections for both IPv4 and IPv6 protocols. I looked around and didn't see a way to override this, either in the mdbc CR, or globally (since the problem is cluster-wide) in the mongodb-kubernetes chart values.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions