Update Configuration Loading (temporalio#8477)

## What Changed

This PR introduces a new `--config-file` flag (and the
`TEMPORAL_SERVER_CONFIG_FILE_PATH ` environment variable) to remove the
dependency on `dockerize` in the Temporal server Docker image.

When a configuration file is specified using either the CLI flag or the
environment variable, the server will load configuration **only** from
that file.
Users who want templating behavior similar to `dockerize` can enable it
by adding the comment `# enable-template` at the top of the
configuration file.

---

### Key Changes

1. **New `--config-file` flag:**

* Adds a global `--config-file` flag that accepts a path to a single
configuration file (absolute or relative to the project root).
* Can also be set via the `TEMPORAL_SERVER_CONFIG_FILE_PATH `
environment variable.

2. **Deprecated legacy flags:**

* The `--config`, `--env`, and `--zone` flags are now marked as
**deprecated** in CLI help text.
   * These flags still work for backward compatibility.

3. **Embedded config template:**

* The `config_template.yaml` file is now embedded in the binary to
support loading configuration from environment variables.
* Templating is supported if the file includes the `# enable-template`
comment at the top.

4. **Templating support:**

* Configuration files can use templating by including `#
enable-template` at the beginning of the YAML file.

---

### Configuration Loading Priority (Highest to Lowest)

1. **`--config-file` specified** → Load that specific file
2. **`--config`, `--env`, or `--zone` specified** → Load from
configuration directory (**deprecated**)
3. **No configuration specified** → Load from embedded template using
environment variables (default)

---

### Expected Behavior

The following examples illustrate how the new configuration loading
logic behaves:

* **Default behavior:**
Running `temporal start` without flags loads configuration from
environment variables only using the embedded template.

* **Using `--config-file`:**
`temporal --config-file=/path/to/config.yaml start` loads configuration
from the specified file path.

* **Using `TEMPORAL_SERVER_CONFIG_FILE_PATH`:**
Setting `TEMPORAL_SERVER_CONFIG_FILE_PATH=/path/to/config.yaml temporal
start` has the same effect as using the flag.

* **Validation and error handling:**
The CLI returns clear error messages when conflicting flags or
environment variables are used, or when a specified file does not exist.
---

## Breaking Change

The default behavior of `temporal start` has changed.
It now loads configuration **from environment variables** instead of
using a default template path.

---------

Co-authored-by: Alex Stanfield <[email protected]>
Co-authored-by: michaely520 <[email protected]>
Co-authored-by: Yichao Yang <[email protected]>
Co-authored-by: David Reiss <[email protected]>

Author: 4 people

CONTRIBUTING.md

@@ -260,9 +260,12 @@ All PR titles should start with Upper case and have no dot at the end.
 ## Go version update
 
 1. In this repository, update `go` in `go.mod`.
-2. In [docker-builds](https://github.com/temporalio/docker-builds/), update the base images:
+2. ~~In [docker-builds](https://github.com/temporalio/docker-builds/), update the base images:
 [base-ci-builder](https://github.com/temporalio/docker-builds/blob/main/docker/base-images/base-ci-builder.Dockerfile)
-and [base-builder](https://github.com/temporalio/docker-builds/blob/main/docker/base-images/base-builder.Dockerfile)
+and [base-builder](https://github.com/temporalio/docker-builds/blob/main/docker/base-images/base-builder.Dockerfile)~~ 
+**Note:** The docker-builds repository is now deprecated and will be archived.
+
+<!-- TODO: Remove docker/config_template.yaml after temporalio/docker-builds repository is archived -->
 
 ## License
 

Makefile

@@ -595,44 +595,44 @@ stop-dependencies-cdc:
 start: start-sqlite
 
 start-cass-es: temporal-server
-	./temporal-server --env development-cass-es --allow-no-auth start
+	./temporal-server --config-file config/development-cass-es.yaml --allow-no-auth start
 
 start-cass-es-dual: temporal-server
-	./temporal-server --env development-cass-es-dual --allow-no-auth start
+	./temporal-server --config-file config/development-cass-es-dual.yaml --allow-no-auth start
 
 start-cass-es-custom: temporal-server
-	./temporal-server --env development-cass-es-custom --allow-no-auth start
+	./temporal-server --config-file config/development-cass-es-custom.yaml --allow-no-auth start
 
 start-es-fi: temporal-server
-	./temporal-server --env development-cass-es-fi --allow-no-auth start
+	./temporal-server --config-file config/development-cass-es-fi.yaml --allow-no-auth start
 
 start-mysql: start-mysql8
 
 start-mysql8: temporal-server
-	./temporal-server --env development-mysql8 --allow-no-auth start
+	./temporal-server --config-file config/development-mysql8.yaml --allow-no-auth start
 
 start-mysql-es: temporal-server
-	./temporal-server --env development-mysql-es --allow-no-auth start
+	./temporal-server --config-file config/development-mysql-es.yaml --allow-no-auth start
 
 start-postgres: start-postgres12
 
 start-postgres12: temporal-server
-	./temporal-server --env development-postgres12 --allow-no-auth start
+	./temporal-server --config-file config/development-postgres12.yaml --allow-no-auth start
 
 start-sqlite: temporal-server
-	./temporal-server --env development-sqlite --allow-no-auth start
+	./temporal-server --config-file config/development-sqlite.yaml --allow-no-auth start
 
 start-sqlite-file: temporal-server
-	./temporal-server --env development-sqlite-file --allow-no-auth start
+	./temporal-server --config-file config/development-sqlite-file.yaml --allow-no-auth start
 
 start-xdc-cluster-a: temporal-server
-	./temporal-server --env development-cluster-a --allow-no-auth start
+	./temporal-server --config-file config/development-cluster-a.yaml --allow-no-auth start
 
 start-xdc-cluster-b: temporal-server
-	./temporal-server --env development-cluster-b --allow-no-auth start
+	./temporal-server --config-file config/development-cluster-b.yaml --allow-no-auth start
 
 start-xdc-cluster-c: temporal-server
-	./temporal-server --env development-cluster-c --allow-no-auth start
+	./temporal-server --config-file config/development-cluster-c.yaml --allow-no-auth start
 
 ##### Grafana #####
 update-dashboards:

cmd/server/main.go

@@ -42,29 +42,34 @@ func buildCLI() *cli.App {
 			Name:    "root",
 			Aliases: []string{"r"},
 			Value:   ".",
-			Usage:   "root directory of execution environment",
+			Usage:   "root directory of execution environment (deprecated)",
 			EnvVars: []string{config.EnvKeyRoot},
 		},
 		&cli.StringFlag{
 			Name:    "config",
 			Aliases: []string{"c"},
 			Value:   "config",
-			Usage:   "config dir path relative to root",
+			Usage:   "config dir path relative to root (deprecated)",
 			EnvVars: []string{config.EnvKeyConfigDir},
 		},
 		&cli.StringFlag{
 			Name:    "env",
 			Aliases: []string{"e"},
 			Value:   "development",
-			Usage:   "runtime environment",
+			Usage:   "runtime environment (deprecated)",
 			EnvVars: []string{config.EnvKeyEnvironment},
 		},
 		&cli.StringFlag{
 			Name:    "zone",
 			Aliases: []string{"az"},
-			Usage:   "availability zone",
+			Usage:   "availability zone (deprecated)",
 			EnvVars: []string{config.EnvKeyAvailabilityZone, config.EnvKeyAvailabilityZoneTypo},
 		},
+		&cli.StringFlag{
+			Name:    "config-file",
+			Usage:   "path to config file (absolute or relative to current working directory)",
+			EnvVars: []string{config.EnvKeyConfigFile},
+		},
 		&cli.BoolFlag{
 			Name:    "allow-no-auth",
 			Usage:   "allow no authorizer",
@@ -105,10 +110,10 @@ func buildCLI() *cli.App {
 			Usage:     "Render server config template",
 			ArgsUsage: " ",
 			Action: func(c *cli.Context) error {
-				cfg, err := config.LoadConfig(
-					c.String("env"),
-					c.String("config"),
-					c.String("zone"),
+				cfg, err := config.Load(
+					config.WithEnv(c.String("env")),
+					config.WithConfigDir(c.String("config")),
+					config.WithZone(c.String("zone")),
 				)
 				if err != nil {
 					return cli.Exit(fmt.Errorf("Unable to load configuration: %w", err), 1)
@@ -139,12 +144,13 @@ func buildCLI() *cli.App {
 				if c.Args().Len() > 0 {
 					return cli.Exit("ERROR: start command doesn't support arguments. Use --service flag instead.", 1)
 				}
+
+				if c.IsSet("config-file") && (c.IsSet("config") || c.IsSet("env") || c.IsSet("zone") || c.IsSet("root")) {
+					return cli.Exit("ERROR: can not use --config, --env, --zone, or --root with --config-file", 1)
+				}
 				return nil
 			},
 			Action: func(c *cli.Context) error {
-				env := c.String("env")
-				zone := c.String("zone")
-				configDir := path.Join(c.String("root"), c.String("config"))
 				services := c.StringSlice("service")
 				allowNoAuth := c.Bool("allow-no-auth")
 
@@ -154,7 +160,22 @@ func buildCLI() *cli.App {
 					services = strings.Split(c.String("services"), ",")
 				}
 
-				cfg, err := config.LoadConfig(env, configDir, zone)
+				var cfg *config.Config
+				var err error
+
+				switch {
+				case c.IsSet("config-file"):
+					cfg, err = config.Load(config.WithConfigFile(c.String("config-file")))
+				case c.IsSet("config") || c.IsSet("env") || c.IsSet("zone"):
+					cfg, err = config.Load(
+						config.WithEnv(c.String("env")),
+						config.WithConfigDir(path.Join(c.String("root"), c.String("config"))),
+						config.WithZone(c.String("zone")),
+					)
+				default:
+					cfg, err = config.Load(config.WithEmbedded())
+				}
+
 				if err != nil {
 					return cli.Exit(fmt.Sprintf("Unable to load configuration: %v.", err), 1)
 				}