Why did the OAP service output such a warning?

[MQPearth]

2023-06-08 08:47:07,627 org.apache.skywalking.oap.server.telemetry.api.HealthCheckMetrics 50 [pool-4-thread-1] WARN [] - Health check fails. reason: can't get itself
2023-06-08 08:47:14,967 org.apache.skywalking.oap.server.telemetry.api.HealthCheckMetrics 50 [pool-7-thread-1] WARN [] - Health check fails. reason: can't get itself
2023-06-08 08:47:14,967 org.apache.skywalking.oap.server.core.storage.ttl.DataTTLKeeperTimer 83 [pool-7-thread-1] INFO [] - The selected first getAddress is 10.244.140.85_11800. The remove stage is skipped.
2023-06-08 08:47:17,628 org.apache.skywalking.oap.server.telemetry.api.HealthCheckMetrics 50 [pool-4-thread-1] WARN [] - Health check fails. reason: can't get itself

My k8s configuration

kind: Deployment
apiVersion: apps/v1
metadata:
  name: skywalking-oap
  namespace: skywalking-oap
  labels: 
    app: skywalking-oap
spec:
  replicas: 2
  selector:
    matchLabels:
      app: skywalking-oap
  template:
    metadata:
      labels: 
        app: skywalking-oap
    spec:
      containers:
      - name: skywalking-oap
        image: apache/skywalking-oap-server:9.4.0-java17
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 11800
          name: "tcp-11800"
        - containerPort: 12800
          name: "tcp-12800"
        env:
        - name: TZ
          value: "Asia/Shanghai"
        - name: SW_STORAGE
          value: "elasticsearch"
        - name: SW_STORAGE_ES_CLUSTER_NODES
          value: "10.11.38.190:9200"
        - name: SW_CLUSTER
          value: "nacos"
        - name: SW_CLUSTER_NACOS_HOST_PORT
          value: "nacos-headless.nacos:8848"
        - name: SW_CLUSTER_NACOS_NAMESPACE
          value: "19a0fa32-ed2e-40f1-a1e1-aae8c81d8cf8"
        - name: SW_CLUSTER_NACOS_USERNAME
          value: "nacos"
        - name: SW_CLUSTER_NACOS_PASSWORD
          value: "nacos"
        - name: SW_CLUSTER_INTERNAL_COM_HOST
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: SW_CLUSTER_INTERNAL_COM_PORT
          value: "11800"
        volumeMounts:
        - name: localtime
          mountPath: /etc/localtime
          readOnly: true
      volumes:
      - name: localtime
        hostPath:
          path: /etc/localtime

[kezhenxu94]

Interesting, you are using Kubernetes but choose to use Nacos instead of the Kubernetes coordinator itself, can you check the Nacos console to see whether the OAP instances are registered and are healthy?

[MQPearth]

[wu-sheng]

I think the conflict is from logical perspective, rather than physical POD unhealth.

Health check fails. reason: can't get itself mean the kernel reading the server IP list from the registration center(Nacos). But it can't match the IP declared in core module(in pod deployment, you most likely would set 0.0.0.0), but in registered IP is 10.244.140.229 which I guess you are using internalComHost.

This is inevitable. Nacos with internalComHost was only added due to out-of-k8s. Choose the k8s native coordinator, please.