Trino 476 with `catalog.management=dynamic`

[thunder-spb]

Hi everyone.

I'm trying to run Trino via HelmChart in Kubernetes (EKS) with dynamic catalogs.

Here is my config for coordinator:

  config.properties: |
    coordinator=true
    node-scheduler.include-coordinator=false
    http-server.http.port=8080
    query.max-memory=4GB
    query.max-memory-per-node=1GB
    discovery.uri=http://localhost:8080
    catalog.management=dynamic
    catalog.prune.update-interval=10s
    catalog.config-dir=/etc/trino/dynamic-catalog

But I'm getting the following error on coordinator...

Errors:
1) Configuration property 'catalog.config-dir' was not used. Did you mean to use 'catalog.store'?

What I miss here?

The path /etc/trino/dynamic-catalog mounted as PVC and this config was working with Trino 450, but not higher...

Was referencing this example: https://github.com/trinodb/charts/pull/178/files#diff-f26c38c9f4d17a97a0c3488c0d93a7a31162bb355101058f50c83c9b6e1eaa58

[nineinchnick]

Are you setting those properties on all nodes, including the workers? Does this error only happen on workers? If yes, you might want to avoid setting catalog.config-dir on workers.

See this for a working example of dynamic catalogs in the Trino Helm chart: https://github.com/trinodb/charts/pull/178/files#diff-f26c38c9f4d17a97a0c3488c0d93a7a31162bb355101058f50c83c9b6e1eaa58

[thunder-spb]

@nineinchnick Workers only has catalog.management=dynamic. This is for coordinator. Workers starting fine. And I was referencing example provided on that link. This config works flawlessly on Trino 450, but not on the latest one...

[nineinchnick]

I just rebased that PR and it works with Trino 476.

I also noticed an existing issue that discusses the same problem, and there are some workarounds there: trinodb/charts#332

[nineinchnick]

I applied the fixes from the issue I mentioned above to trinodb/charts#178 and made sure it actually runs the dynamic catalogs test, as I had rebased it incorrectly earlier. Now I checked the logs to make sure that test passed.

The solution is to put catalog.config-dir in a separate properties file called catalog-store.properties.

[thunder-spb]

@nineinchnick yeah, that works for me as well! Oh, that's really confusing but was working fine in Trino 450... Yes, I know this feature is experemental, but anyway this should be reflected in documentation. prune configuration should be moved to that file as well, because, according to the documentation, it used only for dynamic catalog management...

Thanks anyway!

For those who will come here with the similar issue, in addition to other settings, add this to your helm values.yaml:

    coordinator:
      additionalConfigFiles:
        catalog-store.properties: |
          catalog.config-dir=/etc/trino/dynamic-catalog

Ref: trinodb/charts/issues/332#issuecomment-2873679632

Explanation: trinodb/charts/issues/332#issuecomment-2862232216