Stanza not created and archiving not working on Scaleway Object Storage, but manual command works

[pchovelon]

I tried to use the plugin with Scaleway Object Storage. After creating the Cluster, the first archive command is launched but failed with this error :

"error": "rpc error: code = Unknown desc = stanza creation failed: can't create stanza, stdout: 2025-12-30 09:08:40.035 P00  ERROR: [039]: HTTP request failed with 400 (Bad request):\n                                    *** Path/Query ***:\n                                    HEAD /test-pgbackrest/cluster/archive/main/archive.info\n                                    *** Request Headers ***:\n                                    authorization: <redacted>\n                                    content-length: 0\n                                    host: s3.fr-par.scw.cloud\n                                    x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\n                                    x-amz-date: <redacted>\n                                    *** Response Headers ***:\n                                    cache-control: no-cache\n                                    connection: close\n                                    content-length: 90\n                                    content-type: text/html\n, error : exit status 39"

I changed the logLevel parameter to get the command used. Here is the stanza-create command :

INFO: stanza-create command begin 2.57.0: --exec-id=76-9034f4ec --lock-path=/controller/tmp/pgbackrest-cnpg-plugin.lock --log-level-console=info --log-level-file=off --pg1-path=/var/lib/postgresql/data/pgdata --repo1-path=/cluster --repo1-s3-bucket=test-pgbackrest --repo1-s3-endpoint=s3.fr-par.scw.cloud --repo1-s3-key=<redacted> --repo1-s3-key-secret=<redacted> --repo1-s3-region=fr-par --repo1-s3-uri-style=path --no-repo1-storage-verify-tls --repo1-type=s3 --stanza=main

I tried to execute the same command inside the pgbackrest container while exporting KEY and KEY SECRET in env vars :

k exec -it cluster-1 -c plugin-pgbackrest -- bash
postgres@cluster-1:/$ export PGBACKREST_REPO1_S3_KEY=KEY
postgres@cluster-1:/$ export PGBACKREST_REPO1_S3_KEY_SECRET=SECRET

postgres@cluster-1:/$ pgbackrest stanza-create --lock-path=/controller/tmp/pgbackrest-cnpg-plugin.lock --log-level-console=info --log-level-file=off --pg1-path=/var/lib/postgresql/data/pgdata --repo1-path=/cluster --repo1-s3-bucket=test-pgbackrest --repo1-s3-endpoint=s3.fr-par.scw.cloud --repo1-s3-region=fr-par --repo1-s3-uri-style=path --no-repo1-storage-verify-tls --repo1-type=s3 --stanza=main

And It works !

2025-12-30 09:17:13.999 P00   INFO: stanza-create command begin 2.57.0: --exec-id=114-56dfb629 --lock-path=/controller/tmp/pgbackrest-cnpg-plugin.lock --log-level-console=info --log-level-file=off --pg1-path=/var/lib/postgresql/data/pgdata --repo1-path=/cluster --repo1-s3-bucket=test-pgbackrest --repo1-s3-endpoint=s3.fr-par.scw.cloud --repo1-s3-key=<redacted> --repo1-s3-key-secret=<redacted> --repo1-s3-region=fr-par --repo1-s3-uri-style=path --no-repo1-storage-verify-tls --repo1-type=s3 --stanza=main
2025-12-30 09:17:14.018 P00   INFO: stanza-create for stanza 'main' on repo1
2025-12-30 09:17:16.805 P00   INFO: stanza-create command end: completed successfully (2811ms)

The issue seems to be in the plugin part.

After the manual creation of the stanza, the plugin seems not to take it in account. The error is still the same when trying to archive a WAL :

{
  "level": "error",
  "ts": "2025-12-30T09:20:00.573966356Z",
  "logger": "wal-archive",
  "msg": "Error while calling ArchiveWAL, failing",
  "pluginName": "pgbackrest.dalibo.com",
  "logging_pod": "cluster-1",
  "error": "rpc error: code = Unknown desc = stanza creation failed: can't create stanza, stdout: 2025-12-30 09:20:00.495 P00   INFO: stanza-create command begin 2.57.0: --exec-id=134-efab9deb --lock-path=/controller/tmp/pgbackrest-cnpg-plugin.lock --log-level-console=info --log-level-file=off --pg1-path=/var/lib/postgresql/data/pgdata --repo1-path=/cluster --repo1-s3-bucket=test-pgbackrest --repo1-s3-endpoint=s3.fr-par.scw.cloud --repo1-s3-key=<redacted> --repo1-s3-key-secret=<redacted> --repo1-s3-region=fr-par --repo1-s3-uri-style=path --no-repo1-storage-verify-tls --repo1-type=s3 --stanza=main\n2025-12-30 09:20:00.514 P00   INFO: stanza-create for stanza 'main' on repo1\n2025-12-30 09:20:00.570 P00  ERROR: [039]: HTTP request failed with 400 (Bad request):\n                                    *** Path/Query ***:\n                                    HEAD /test-pgbackrest/cluster/archive/main/archive.info\n                                    *** Request Headers ***:\n                                    authorization: <redacted>\n                                    content-length: 0\n                                    host: s3.fr-par.scw.cloud\n                                    x-amz-content-sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855\n                                    x-amz-date: <redacted>\n                                    *** Response Headers ***:\n                                    cache-control: no-cache\n                                    connection: close\n                                    content-length: 90\n                                    content-type: text/html\n2025-12-30 09:20:00.570 P00   INFO: stanza-create command end: aborted with exception [039]\n, error : exit status 39",
  "stacktrace": "github.com/cloudnative-pg/machinery/pkg/log.(*logger).Error\n\tpkg/mod/github.com/cloudnative-pg/machinery@v0.3.1/pkg/log/log.go:125\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cnpi/plugin/client.(*data).innerArchiveWAL\n\tinternal/cnpi/plugin/client/wal.go:77\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cnpi/plugin/client.(*data).ArchiveWAL\n\tinternal/cnpi/plugin/client/wal.go:39\ngithub.com/cloudnative-pg/cloudnative-pg/pkg/management/postgres/archiver.archiveWALViaPlugins\n\tpkg/management/postgres/archiver/archiver.go:304\ngithub.com/cloudnative-pg/cloudnative-pg/pkg/management/postgres/archiver.internalRun\n\tpkg/management/postgres/archiver/archiver.go:168\ngithub.com/cloudnative-pg/cloudnative-pg/pkg/management/postgres/archiver.Run\n\tpkg/management/postgres/archiver/archiver.go:153\ngithub.com/cloudnative-pg/cloudnative-pg/internal/cmd/manager/walarchive.NewCmd.func1\n\tinternal/cmd/manager/walarchive/cmd.go:65\ngithub.com/spf13/cobra.(*Command).execute\n\tpkg/mod/github.com/spf13/cobra@v1.10.1/command.go:1015\ngithub.com/spf13/cobra.(*Command).ExecuteC\n\tpkg/mod/github.com/spf13/cobra@v1.10.1/command.go:1148\ngithub.com/spf13/cobra.(*Command).Execute\n\tpkg/mod/github.com/spf13/cobra@v1.10.1/command.go:1071\nmain.main\n\tcmd/manager/main.go:71\nruntime.main\n\t/opt/hostedtoolcache/go/1.25.5/x64/src/runtime/proc.go:285"
}

Here are the YAML file used :

---
apiVersion: v1
kind: Secret
metadata:
  name: pgbackrest-s3-secret
type: Opaque
data:
  ACCESS_KEY_ID: xxxxx
  ACCESS_REGION: xxxxx
  ACCESS_SECRET_KEY: xxxxx
---
apiVersion: pgbackrest.dalibo.com/v1
kind: Stanza
metadata:
  name: stanza-sample
spec:
  stanzaConfiguration:
    name: main
    s3Repositories:
      - bucket: pgbackrest
        endpoint: s3.fr-par.scw.cloud
        region: fr-par
        repoPath: /cluster
        uriStyle: path
        verifyTLS: false
        retentionPolicy:
          full: 3
          fullType: count
          diff: 2
          archive: 2
          archiveType: full
          history: 30
        secretRef:
          accessKeyId:
            name: pgbackrest-s3-secret
            key: ACCESS_KEY_ID
          secretAccessKey:
            name: pgbackrest-s3-secret
            key: ACCESS_SECRET_KEY
---
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster
spec:
  imagePullPolicy: IfNotPresent
  instances: 1
  plugins:
    - name: pgbackrest.dalibo.com
      parameters:
        stanzaRef: stanza-sample
      isWALArchiver: true
  storage:
    size: 1Gi