Coolify deployment with flexible storage

[SpuriousGer]

Has someone made this work with Coolify in a manner that supports multiple yaml files in the persistent storage part of coolify?

Ideally, I would be able to create new pipelines by adding files there.

I am struggling and might be missing something really obvious at the moment.

[mihaitodor]

Hey @SpuriousGer I wonder if Streams Mode might help. Unfortunately, the builtin watcher mode (see redpanda-connect streams --help) doesn't support loading new files dynamically, but there is a REST API which you can use to create new pipelines. Feel free to raise an issue on the Benthos repo if you'd like to have Connect dynamically pick up new files and we can look into supporting it eventually.