[ISSUE] External Keycloak with predefined identity secrets results in invalid_client_credentials

[ggrames]

Describe the issue:

External Keycloak reports invalid_client_credentials when i use the predefined secret for zeebe.

I have set in the values.yaml file the usage of an external Keycloak with the version: 23.0.7
Also i have defined my secrets for the clients in the identity-secret-for-components in my platform before install via helm.
The secret looks like:

apiVersion: v1
kind: Secret
metadata:
  name: identity-secret-for-components
type: Opaque
data:
  # Identity apps auth.
  connectors-secret: <MY BAS64 Encoded Password for the client>
  operate-secret: <MY BAS64 Encoded Password for the client>
  optimize-secret: <MY BAS64 Encoded Password for the client>
  tasklist-secret: <MY BAS64 Encoded Password for the client>
  zeebe-secret: <MY BAS64 Encoded Password for the client>
  console-secret: <MY BAS64 Encoded Password for the client>
  # First user password
  identity-firstuser-password: <MY BAS64 Encoded Password for the client>

I make always a new installation, because i evaluate the product with following command:

helm install camunda-platform camunda/camunda-platform --skip-crds --version=11.0.1 -f values.yaml  --set global.compatibility.openshift.adaptSecurityContext=force

Actual behavior:

When i use the settings like described above (with the predefined secret for the zeebe client), the external Keycloak throws an error during the login to the zeebe Gateway with the message:

2024-10-17 08:17:28,726 WARN  [org.keycloak.events] (executor-thread-1435) type="CLIENT_LOGIN_ERROR", realmId="camunda-platform", clientId="zeebe", userId="null", ipAddress="123.456.789.123", error="invalid_client_credentials", grant_type="client_credentials"

I can see the created client in the keycloak with the correct password from the secret (identity-secret-for-components)
I test the connection with the camunda modeler - deploy (rocket) button in the Desktop Application.
Camunda Modeler logs following error:

HTTPError: Response code 401 (Unauthorized)
    at Request.<anonymous> (c:\Daten\Development\camunda-modeler-5.28.0-win-x64\resources\app.asar\node_modules\got\dist\source\as-promise\index.js:118:42)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5) {
  code: 'ERR_NON_2XX_3XX_RESPONSE'

Expected behavior:

The Camunda Modeler should deploy the model and the client should authenticate against the keycloak without problems.

How to reproduce:
It is always reproduceable when i set the value in the values.yaml for the predefinde Secret

global:
  identity:
    auth:
      ## @param global.identity.auth.enabled if true, enables the identity authentication otherwise basic-auth will be used on all services.
      enabled: true
      zeebe:
        ## @param global.identity.auth.zeebe.clientId defines the client id, which is used by Zeebe in authentication flows.
        clientId: zeebe
        ## @param global.identity.auth.zeebe.existingSecret can be used to use an own existing secret. If not set a random secret is generated.
        existingSecret:
          name: identity-secret-for-components
        ## @param global.identity.auth.zeebe.existingSecretKey defines the key within the existing secret object.
        existingSecretKey: zeebe-secret
        ## @param global.identity.auth.zeebe.audience defines the audience, which is used by Zeebe.
        audience: zeebe-api
        ## @param global.identity.auth.zeebe.tokenScope defines the token scope, which is used by Zeebe.
        tokenScope:

It is working
It works when i delete the option to take the zeebe client secret from the automatic generate secret from the helm chart.
Then my external keycloak is also happy and my test with the camunda modeler runs without any problem and i can deploy to the camunda platform.
In the working scenario the values.xml looks like:

global:
  identity:
    auth:
      ## @param global.identity.auth.enabled if true, enables the identity authentication otherwise basic-auth will be used on all services.
      enabled: true
      zeebe:
        ## @param global.identity.auth.zeebe.clientId defines the client id, which is used by Zeebe in authentication flows.
        clientId: zeebe
        ## @param global.identity.auth.zeebe.existingSecret can be used to use an own existing secret. If not set a random secret is generated.
        existingSecret:
          #name: identity-secret-for-components
        ## @param global.identity.auth.zeebe.existingSecretKey defines the key within the existing secret object.
        existingSecretKey: zeebe-secret
        ## @param global.identity.auth.zeebe.audience defines the audience, which is used by Zeebe.
        audience: zeebe-api
        ## @param global.identity.auth.zeebe.tokenScope defines the token scope, which is used by Zeebe.
        tokenScope:

Logs:

Environment:

Please note: Without the following info, it's hard to resolve the issue and probably it will be closed.

• Platform: OKD 4.15
• Helm CLI version: version.BuildInfo{Version:"v3.14.3", GitCommit:"f03cc04caaa8f6d7c3e67cf918929150cf6f3f12", GitTreeState:"clean", GoVersion:"go1.21.7"}
• Chart version: 11.0.1
• Values file: see the snippets in the reproducable