Adapt configuration file structure, tooling (start scripts), config option readers (Java) and docs for new bind.listeners configuration option

[michaelkoepf]

Purpose

Linked issue: close #618 close #619

With the introduction of bind.listeners (and, subsequently, also Kafka compatibility), adaptions to the structure of Fluss' config files are necessary. In particular, we need to introduce separate, dedicated configuration files for coordinator and tablet servers. These adaptions also require changes in the CLI tooling (start scripts) and Java config option readers. Backward compatibility should be maintained. In addition, docs need to be adapted.

As changes of #618 and #619 are tightly coupled, they are resolved within one PR.

Brief change log

Modules

• fluss-dist

  • Remove server.yaml and introduce a new common.yaml. common.yaml is the common configuration file read by all start scripts and all Java applications that previously read from server.yaml. common.yaml is introduced to better reflect the purpose of this configuration file in its name, as it is read by many different scripts and modules, incl. all start scripts, the Fluss server module and also the Lakehouse CLI. server.yaml is deprecated and should be removed when support for the [coordinator|tablet-server].host config option is dropped.
  • Adapt config.sh to read from common.yaml but ensure backward compatibility with the deprecated server.yaml. Constrain it to either use the deprecated server.yaml configuration file or the new common.yaml file; do not allow mixed configurations.
  • Add dedicated configuration files for tablet and coordinator server to support instance-specific configurations. This includes, e.g., bind.listeners or Kafka ports.

• fluss-common:

  • Adapt src/main/java/com/alibaba/fluss/config/GlobalConfiguration#loadConfiguration s.t. it can load the new configuration file structure, i.e., common.yaml and additional dedicated configuration files (e.g., coordinator-server.yaml). Maintain backward compatibility for old configuration file structure (server.yaml).

• fluss-server:

  • Adaptions to pass through server type. Choose dedicated default configuration file that should be loaded depending on the server type.

Docker

• Append content of FLUSS_PROPERTIES environment variable to common and instance-specific configuration files.
• Ensure backward compatability by removing bind.listeners from YAML config files.

Docs

• Update docs for new bind.listeners config option.
• Update sections that reference server.yaml.

Tests

Unit Tests

• Added new test class fluss-common/src/test/java/com/alibaba/fluss/config/GlobalConfigurationTest.java
• Adapted and added tests in fluss-server/src/test/java/com/alibaba/fluss/server/utils/ConfigurationParserUtilsTest.java

Start Scripts

Verified manually by running commands from docs

API and Format

n/a

Documentation

n/a

[michaelkoepf]

Note for reviewer: I suggest to rename server.yaml to common.yaml from 0.7.0 onward for the following reasons.

1. The file name server is a bit misleading, because the config file is not only read by the server but also by config.sh and the lakehouse module. And it's easier to change it now than at some later point.
2. It avoids confusion. With renaming it, there is a clear separation and it is clear which config files are read. It is either server.yaml or common.yaml + depending on the module, an additional file ([coordinator-server|tablet-server].yaml).

We can remove server.yaml when we deprecate the old [coordinator|tablet-server]-host config options.

Of course, I am open to discuss if renaming the file is a good approach or not.