Lifecycle's analysis raises an error during lifecycle when dockercfgjson file includes auths with FQ Image name

[cmoulliard]

Summary

Lifecycle's analysis and google-cointainer, when executed in a kubernetes cluster where the dockercfgjson file is mounted from a secret, raises an error during analyse step:

Error checking read/write access: POST https://quay.io/v2/ch007m/workloads/a/b/blobs/uploads/: UNAUTHORIZED: access to the requested resource is not authorized; map[]
ERROR: failed to initialize analyzer: validating registry write access: ensure registry read/write access to [quay.io/ch007m/workloads/a/b](http://quay.io/ch007m/workloads/a/b)

This error occurs when the dockercfgjson file includes auths with FQ Image name.

Lifecycle will not fail using

{
  "auths": {
    "quay.io": {
      "auth": "Y2gw...MQ==",
      "email": ""
    }
  }
}

or

{
  "auths": {
    "quay.io/ch007m/workloads/a/b": {
      "auth": "Y2gwM...XMQ==",
      "email": ""
    }
  }
}

but will fail here

{
  "auths": {
    "quay.io/ch007m/workloads/a/a": {
      "auth": "Y2gwM...NjJPMg==",
      "email": ""
    },
    "quay.io/ch007m/workloads/a/b": {
      "auth": "Y2gwMD...MQ==",
      "email": ""
    }
  }
}

Reproduction

To reproduce, create a json auths file containing fully qualified image names

{
  "auths": {
    "quay.io/ch007m/workloads/a/a": {
      "auth": "Y2gwMDd....NjJPMg==",
      "email": ""
    },
    "quay.io/ch007m/workloads/a/b": {
      "auth": "Y2gwM...zRXMQ==",
      "email": ""
    }
  }
}

Next, create a kubernetes secret and deploy it on the cluster

You can install Tekton, Dashboard on a k8s cluster using the following bash script and pass as arg the json file containing the auths

AUTHS_FILE=$1

BUILDER_IMAGE=paketobuildpacks/builder-jammy-tiny:0.0.176
LIFECYCLE_IMAGE=buildpacksio/lifecycle:0.17.0
RUN_IMAGE=paketobuildpacks/run-jammy-tiny:latest

IMAGE_NAME=quay.io/ch007m/workloads/a/b

echo "### Installing Tekton and Dashboard"
kubectl apply -f https://github.com/tektoncd/pipeline/releases/download/v0.48.0/release.yaml
kubectl apply -f https://storage.googleapis.com/tekton-releases/dashboard/latest/release.yaml

VM_IP=127.0.0.1
kubectl create ingress tekton-ui -n tekton-pipelines --class=nginx --rule="tekton-ui.$VM_IP.nip.io/*=tekton-dashboard:9097"

kubectl apply -f https://raw.githubusercontent.com/tektoncd/catalog/main/task/git-clone/0.9/git-clone.yaml
kubectl apply -f https://raw.githubusercontent.com/redhat-buildpacks/testing/baa8cb39f92818f48e770bd948841a37412944f1/k8s/tekton/buildpacks-phases.yml

echo "##### Testing lifecycle with: "
cat /Users/cmoullia/code/redhat-buildpacks/testing/_tmp/$AUTHS_FILE

kubectl delete secret/dockercfg-dummy
kubectl create secret generic dockercfg-dummy --from-file=.dockerconfigjson=$AUTHS_FILE --type=kubernetes.io/dockerconfigjson

kubectl delete PipelineRun/buildpacks-phases
kubectl delete pvc/ws-pvc
kubectl delete sa/sa-with-secrets

cat <<EOF | kubectl apply -f -
---
apiVersion: v1
imagePullSecrets:
- name: dockercfg-dummy
kind: ServiceAccount
metadata:
  name: sa-with-secrets
secrets:
- name: dockercfg-dummy
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: ws-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Mi
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: buildpacks-phases
  labels:
    app.kubernetes.io/description: "Buildpacks-PipelineRun"
spec:
  serviceAccountName: sa-with-secrets
  pipelineSpec:
    workspaces:
      - name: source-ws
      - name: cache-ws
    tasks:
      - name: fetch-repository
        taskRef:
          name: git-clone
        workspaces:
          - name: output
            workspace: source-ws
        params:
          - name: url
            value: https://github.com/quarkusio/quarkus-quickstarts.git
          - name: deleteExisting
            value: "true"
      - name: buildpacks
        taskRef:
          name: buildpacks-phases
        runAfter:
          - fetch-repository
        workspaces:
          - name: source
            workspace: source-ws
          - name: cache
            workspace: cache-ws
        params:
          - name: APP_IMAGE
            value: ${IMAGE_NAME}
          - name: SOURCE_SUBPATH
            value: getting-started
          - name: CNB_BUILDER_IMAGE
            value: ${BUILDER_IMAGE}
          - name: CNB_LIFECYCLE_IMAGE
            value: ${LIFECYCLE_IMAGE}
          - name: RUN_IMAGE
            value: ${RUN_IMAGE}
          - name: ENV_VARS
            value:
              - BP_NATIVE_IMAGE=false
              - BP_MAVEN_BUILT_ARTIFACT=target/quarkus-app/lib/ target/quarkus-app/*.jar target/quarkus-app/app/ target/quarkus-app/quarkus/
              - BP_MAVEN_BUILD_ARGUMENTS=package -DskipTests=true -Dmaven.javadoc.skip=true -Dquarkus.package.type=fast-jar
  workspaces:
    - name: source-ws
      subPath: source
      persistentVolumeClaim:
        claimName: ws-pvc
    - name: cache-ws
      subPath: cache
      persistentVolumeClaim:
        claimName: ws-pvc
EOF

tkn pipelinerun logs buildpacks-phases -f

[natalieparellano]

The failure is likely stemming from failure to find an auth here:

lifecycle/auth/keychain.go

Line 249 in 105808c

auth, err = keychain.Resolve(r.Context().Registry)

We would need to investigate how this is being done in GGCR and if it's possible to push a change upstream there.