[ISSUE] Connectors not working when identity is disabled and inbound.mode set to credentials

[p4tr0ckz]

Describe the issue:

When using a minimalistic setup and all identity features disabled but connectors inbound mode with credentials enabled, connectors doesnt work.

Actual behavior:

In our current Setup we're only using the minimalistic values (provided here).
But since we also need connectors to allow inbound connections, i tried the following value settings:

connectors:
  enabled: true
  inbound:
    mode: credentials

This enables connectors inbound, but it also sets the CAMUNDA_OPERATE_CLIENT_USERNAME to connectors. This leads to java.lang.RuntimeException: Unable to authenticate since operate Could not find user with userId 'connectors'..

Logs snippet from connectors

2024-09-17T13:36:12.914Z ERROR 1 --- [   scheduling-1] i.c.common.auth.SimpleAuthentication     : Authenticating for OPERATE failed due to java.lang.RuntimeException: Unable to authenticate due to missing Set-Cookie
2024-09-17T13:36:12.914Z ERROR 1 --- [   scheduling-1] i.c.c.r.i.i.ProcessDefinitionImporter    : Failed to import process definitions

java.lang.RuntimeException: Unable to authenticate
  at io.camunda.common.auth.SimpleAuthentication.retrieveToken(SimpleAuthentication.java:58)
  at io.camunda.common.auth.SimpleAuthentication.getTokenHeader(SimpleAuthentication.java:79)
  at io.camunda.common.http.DefaultHttpClient.retrieveToken(DefaultHttpClient.java:207)
  at io.camunda.common.http.DefaultHttpClient.post(DefaultHttpClient.java:148)
  at io.camunda.operate.CamundaOperateClient.searchProcessDefinitionResults(CamundaOperateClient.java:46)
  at io.camunda.connector.runtime.inbound.importer.ProcessDefinitionSearch.query(ProcessDefinitionSearch.java:72)
  at io.camunda.connector.runtime.inbound.importer.ProcessDefinitionImporter.scheduleImport(ProcessDefinitionImporter.java:55)
  at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source)
  at java.base/java.lang.reflect.Method.invoke(Unknown Source)
  at org.springframework.scheduling.support.ScheduledMethodRunnable.runInternal(ScheduledMethodRunnable.java:130)
  at org.springframework.scheduling.support.ScheduledMethodRunnable.lambda$run$2(ScheduledMethodRunnable.java:124)
  at io.micrometer.observation.Observation.observe(Observation.java:499)
  at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:124)
  at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
  at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
  at java.base/java.util.concurrent.FutureTask.runAndReset(Unknown Source)
  at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
  at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
  at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
  at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: Unable to authenticate due to missing Set-Cookie
  at io.camunda.common.auth.SimpleAuthentication.retrieveToken(SimpleAuthentication.java:53)
  ... 19 common frames omitted

Logs snippet from operate

2024-09-17 13:36:16.968 ERROR 7 --- [nio-8080-exec-4] w.a.UsernamePasswordAuthenticationFilter : An internal error occurred while trying to authenticate the user.

org.springframework.security.authentication.InternalAuthenticationServiceException: Could not find user with userId 'connectors'.
	at org.springframework.security.authentication.dao.DaoAuthenticationProvider.retrieveUser(DaoAuthenticationProvider.java:118) ~[spring-security-core-6.2.5.jar!/:6.2.5]
	at org.springframework.security.authentication.dao.AbstractUserDetailsAuthenticationProvider.authenticate(AbstractUserDetailsAuthenticationProvider.java:133) ~[spring-security-core-6.2.5.jar!/:6.2.5]
[....]
	at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]
Caused by: io.camunda.operate.store.NotFoundException: Could not find user with userId 'connectors'.
	at io.camunda.operate.store.elasticsearch.ElasticsearchUserStore.getById(ElasticsearchUserStore.java:87) ~[operate-schema-8.5.6.jar!/:8.5.6]
	at io.camunda.operate.webapp.security.auth.OperateUserDetailsService.loadUserByUsername(OperateUserDetailsService.java:108) ~[!/:8.5.6]
	at io.camunda.operate.webapp.security.auth.OperateUserDetailsService.loadUserByUsername(OperateUserDetailsService.java:39) ~[!/:8.5.6]
	at io.camunda.operate.webapp.security.auth.OperateUserDetailsService$$SpringCGLIB$$0.loadUserByUsername(<generated>) ~[!/:8.5.6]
	at org.springframework.security.authentication.dao.DaoAuthenticationProvider.retrieveUser(DaoAuthenticationProvider.java:103) ~[spring-security-core-6.2.5.jar!/:6.2.5]
	... 97 more

I cant use global.identity.auth.connectors.clientId since Identity is disabled.

I tried overwriting it with the following, but that doesnt work, since the username is hardcoded here

connectors:
  enabled: true
  inbound:
    mode: credentials
  auth:
    existingSecretKey: demo
  env:
    - name: CAMUNDA_OPERATE_CLIENT_USERNAME
      value: "demo"  

Expected behavior:

I would suggest the possibility to overwrite the username within the configmap:

      {{- if or (eq .Values.connectors.inbound.mode "credentials") (and .Values.global.identity.auth.enabled (eq .Values.connectors.inbound.mode "oauth") ) }}
      operate:
        client:
          url: {{ include "camundaPlatform.operateURL" . | quote }}
          {{- if eq .Values.connectors.inbound.mode "credentials" }}
          username: {{ .Values.connectors.inbound.auth.clientId | default "connectors" }}
          {{- end }}
      {{- end }}

or extend the helpers.tpl

{{/*
[connectors] Defines the auth client
*/}}
{{- define "connectors.authClientId" -}}
  {{- if .Values.global.identity.auth.enabled -}}
    {{- .Values.global.identity.auth.connectors.clientId -}}
  {{- else -}}
    {{- .Values.connectors.inbound.auth.clientId | default "connectors" -}}
  {{- end -}}
{{- end }}

How to reproduce:

• Install latest camunda-platform Chart
• Use minimal values.yaml (provided here)
• change connectors.inbound.mode to credentials

Environment:

Please note: Without the following info, it's hard to resolve the issue and probably it will be closed.

• Platform: local kind & AKS
• Helm CLI version: 3.15.1
• Chart version: 10.4.0
• Values file: minimal

fyi, my current solution is:
Using a local git clone of Chart Version 9.2.0, changed this line in connectors deployment.yaml to 'demo'.
Tried this in a bunch of other versions (9.4.0, 10.3.0, 10.4.0) as well, but it didnt work within the configmap (which are used since 10.0) and i'm kind out of ideas.

[jessesimpson36]

possible duplicate of
#2334

update: not a duplicate, the linked issue did not have inbound.mode set to credentials and was the reason for it's failure.

[hisImminence]

We got an internal request on this ticket:

Any advise on how to configure or work around this?

[hisImminence]

@sbuettner could you please have a look on this issue and see if the proposed configuration is valid for using the connectors?

[sbuettner]

@chillleader was this maybe fixed by your HELM chart update?

[chillleader]

No, this seems unrelated to my recent fix.

From the Connectors side: the issue makes sense and the proposed solution also looks good. The only thing I'd reason about is naming of this parameter in the Helm chart - since this is technically not an OAuth client ID, maybe we can call it {{- .Values.connectors.inbound.auth.username }}. But no strong preference here, happy to leave this up the the assignee.

Another note: starting from 8.8, the Connectors will no longer need Operate client properties (due to switching entirely to Camunda REST API), so this param is only needed for 8.7 and below.

[hisImminence]

Thanks @chillleader! Then we should be fine and can work on reproducing and fixing it.