ERROR: "The node does not support INCREMENTAL_ALTER_CONFIGS" when trying to update a topic config

[laertis]

Problem

It appears that the new Kafka Lenses HQ (>6.0) does not support INCREMENTAL_ALTER_CONFIGS when trying to update a Kafka Topic config for a Kafka cluster with version < 4.0.0

Root Cause

Reading through https://kafka.apache.org/42/getting-started/upgrade/ It looks like form kafka version 4.0.0 and on incrementalAlterConfigs API is used to alter broker configurations instead of alterConfigs API which has been deprecated

kafka-configs.sh now uses incrementalAlterConfigs API to alter broker configurations instead of the deprecated alterConfigs API, and it will fall directly if the broker doesn’t support incrementalAlterConfigs API, which means the broker version is prior to 2.3.x. See KIP-1011 for more details.

Extra Configuration Attempt

Based on Kafka Lenses 5.5 documentation I tried the following configuration parameters

locals {
  lenses_agent_append_conf = <<EOT
    lenses.kafka.settings.client.api.version.request = false
    lenses.kafka.settings.client.broker.version.fallback = "2.1.0"
EOT
}

which I then tried to write to /data/lenses-agent.conf through environment variables using both (but on different tries)

     environment = [
        {
          name  = "LENSES_AGENT_APPEND_CONF"
          value = local.lenses_agent_append_conf
        }
      ]

and on another try

     environment = [
        {
          name  = "LENSES_APPEND_CONF"
          value = local.lenses_agent_append_conf
        },
        {
          name  = "EXPERIMENTAL"
          value = TRUE
        }
      ]

But they both failed with

Lenses failed to start...java.lang.IllegalArgumentException: Old configuration keys detected. Please remove wrong keys and restart Lenses.Configuration keys that are not being handled anymore:
- lenses.kafka.settings

Suggested Solution

Would it be possible to expose a config that would allow Lenses to use an older Kafka client (<4.0.0) that allows this ?
This will provide support for Kafka cluster with version < 4.0 which are quite many

Thank you !