Skip to content

Attu pod restarts when changing the database #648

Open
@iurii-stepanov

Description

@iurii-stepanov
  • Milvus version: 2.4.10
  • Attu version: 2.4.7
  • Deployment mode(standalone or cluster): cluster
  • MQ type(rocksmq, pulsar or kafka): external kafka

Describe the bug:
3 days ago I updated Milvus from 2.3.1 to 2.4.10 by changing images for Milvus, Attu and config-tool.
Sometimes when I change the DB in WebIU Attu pod restarts with message in the log bellow.
The thing is that collection "beauty_filtered" really isn't in database "cv_search". This collection in the "default" DB.

Steps to reproduce:

  1. Login to Attu
  2. Change the BD from default to any other

Attu log:

 GET 200 /api/v1/collections 4.417 ms @ Thu, 12 Sep 2024 06:36:52 GMT ::ffff:10.97.47.24 from undefined Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36

/app/dist/src/crons/crons.service.js:74
                    throw new Error(error);
                          ^

Error: can't find collection[database=cv_search][collection=beauty_filtered]
    at CronsService.<anonymous> (/app/dist/src/crons/crons.service.js:74:27)
    at Generator.throw (<anonymous>)
    at rejected (/app/dist/src/crons/crons.service.js:6:65)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Node.js v18.20.4
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Attu deployment spec:

spec:
  replicas: 1
  selector:
    matchLabels:
      app: milvus-attu
      app.kubernetes.io/instance: platform-milvus
      app.kubernetes.io/name: milvus
      component: attu
      release: platform-milvus
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: milvus-attu
        app.kubernetes.io/instance: platform-milvus
        app.kubernetes.io/name: milvus
        component: attu
        release: platform-milvus
      annotations:
        checksum/config: 406cd24a25e78dcb25870e6653055e35f8942eece05dce0a02ade82ebbc6b6cb
        kubectl.kubernetes.io/restartedAt: '2024-09-12T08:37:35Z'
        platform/env: prod
        platform/logs: enabled
        platform/system: milvus-attu
    spec:
      containers:
        - name: attu
          image: >-
            docker-hosted.artifactory/platform/base-images/milvus-attu:v2.4.7
          ports:
            - name: attu
              containerPort: 3000
              protocol: TCP
          env:
            - name: COMPONENT
              value: attu
            - name: MILVUS_URL
              value: http://platform-milvus:19530
          resources:
            limits:
              cpu: 500m
              memory: 512Mi
            requests:
              cpu: 100m
              memory: 256Mi
          livenessProbe:
            httpGet:
              path: /api/v1/healthy
              port: attu
              scheme: HTTP
            initialDelaySeconds: 90
            timeoutSeconds: 5
            periodSeconds: 30
            successThreshold: 1
            failureThreshold: 5
          readinessProbe:
            httpGet:
              path: /api/v1/healthy
              port: attu
              scheme: HTTP
            initialDelaySeconds: 90
            timeoutSeconds: 5
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 5
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          imagePullPolicy: IfNotPresent
          securityContext:
            capabilities:
              drop:
                - ALL
            runAsUser: 0
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      dnsPolicy: ClusterFirst
      securityContext: {}
      imagePullSecrets:
        - name: artifactory-cred-secret
      schedulerName: default-scheduler
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
      maxSurge: 0
  revisionHistoryLimit: 10
  progressDeadlineSeconds: 600

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions