Skip to content

Plugin Panicked error when creating backup #3

@aleablu

Description

@aleablu

Hi guys,
I'm trying to backup my k8s cluster on linode to an aws s3 bucket, I have quite a complex architecture.
Here is the output of velero backup describe:

Name:         visidea
Namespace:    velero
Labels:       velero.io/storage-location=default
Annotations:  velero.io/resource-timeout=10m0s
              velero.io/source-cluster-k8s-gitversion=v1.26.9
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=26

Phase:  PartiallyFailed (run `velero backup logs visidea` for more information)


Warnings:
  Velero:    error: /the server is currently unable to handle the request
             error: /the server is currently unable to handle the request
  Cluster:    <none>
  Namespaces: <none>

Errors:
  Velero:   
             name: /elasticsearch-es-data-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /elasticsearch-es-data-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /elasticsearch-es-data-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /elasticsearch-es-master-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /ir-crawler-to-k8s-68558887cb-lzv4p error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /mongobackup-859c85666f-snsqw error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /mongodb-api-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /mongodb-api-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
             name: /mongodb-api-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
            
             name: /mongodb-harness-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /mongodb-harness-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /mongodb-harness-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /recommendify-apish-7574c4c55f-9fdq4 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /recommendify-cronjobs-crawl-28334355-47cqd error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /data-my-release-etcd-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /data-my-release-etcd-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /data-my-release-etcd-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /datadir-mongodb-ir-0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /datadir-mongodb-ir-1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
             name: /datadir-mongodb-ir-2 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
            
             name: /mongo-claim0 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
            
            
             name: /mongo-claim1 error: /error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference
  Cluster:    <none>
  Namespaces: <none>

Namespaces:
  Included:  default
  Excluded:  <none>

Resources:
  Included:        *
  Excluded:        <none>
  Cluster-scoped:  auto

Label selector:  <none>

Or label selector:  <none>

Storage Location:  default

Velero-Native Snapshot PVs:  auto
Snapshot Move Data:          false
Data Mover:                  velero

TTL:  720h0m0s

CSISnapshotTimeout:    20m0s
ItemOperationTimeout:  4h0m0s

Hooks:  <none>

Backup Format Version:  1.1.0

Started:    2023-11-15 17:47:16 +0100 CET
Completed:  2023-11-15 17:47:30 +0100 CET

Expiration:  2023-12-15 17:47:16 +0100 CET

Total items to be backed up:  427
Items backed up:              427

Velero-Native Snapshots:  0 of 22 snapshots completed successfully (specify --details for more information)

I am currently installing velero via this command:

PLUGIN="velero/velero-plugin-for-aws:v1.0.1,displague/velero-plugin-linode:v0.0.1"

velero install \
    --provider aws \
    --bucket $BUCKET \
    --secret-file $SECRET\
    --backup-location-config region=$REGION,s3ForcePathStyle="true"\
    --use-volume-snapshots=false \
    --plugins $PLUGIN \

I'm using version 1.12 of velero.

Versions of kubernetes are as follows:

  • Client Version: v1.28.1
  • Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
  • Server Version: v1.26.9

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions