Support environment configuration #1849

cretz · 2025-02-25T14:45:25Z

What was changed

Added go.temporal.io/sdk/contrib/envconfig module that supports loading
- Supports conversion to/from toml
- Supports file loading and env var overrides
- Supports loading client options

See temporalio/features#441, this is mostly based on that proposal with some slight changes.

This SDK may support a few more shortcuts than other SDKs due to its use in CLI. Documentation about file format and which env vars do what will be made available on docs.temporal.io before this launches GA.

Checklist

Closes Environment configuration #1815

Fixes temporalio#1815

contrib/envconfig/client_config.go

Sushisource · 2025-02-25T17:14:56Z

contrib/envconfig/client_config.go

+// ClientConfigCodec is codec configuration for a client.
+//
+// WARNING: Environment configuration is currently experimental.
+type ClientConfigCodec struct {


Maybe call this ClientConfigRemoteCodec? What, if anything, could ever be configured here for local codec? And, even then, maybe it'd make sense to have them be different sub-configs

It is the codec field in config and TEMPORAL_CODEC_ prefixed env vars and Codec field name in the structure in addition to this type name. So I want it to match those. If we want though, I could change the config field name, env var prefix, struct field name, and this type name.

Sushisource · 2025-02-25T17:21:46Z

contrib/envconfig/client_config_load.go

+// See [ClientConfig.FromTOML] for details on format.
+//
+// WARNING: Environment configuration is currently experimental.
+func LoadClientConfig(options LoadClientConfigOptions) (ClientConfig, error) {


This and LoadClientOptions don't have distinct enough names, IMO.

Maybe LoadClientOptionsFromFile ?

I think it's distinct enough given its return type and such, but I can change and change the param structure to LoadClientConfigFromFileOptions. Would like to wait on other opinions if anyone has any. I am concerned there may also be too-similar naming concerns in other SDKs.

Sushisource · 2025-02-25T17:26:06Z

contrib/envconfig/client_config_load.go

+	if env == nil {
+		env = EnvLookupOS
+	}
+	if s, ok := env.LookupEnv("TEMPORAL_ADDRESS"); ok {


I think it probably makes sense to make these all public constants. It helps with both documentation and, since people can technically implement their own lookup, they might want to programmatically use them.

I don't expect this to be where such things are documented. I am expecting docs.temporal.io to house the documentation with these environment variables. I don't expect anyone to ever reference such a constant programmatically either like one might expect with an exported constant.

Why not? They could be doing it as part of some infra-as-code stuff.

Even more obviously, our own CLI PR could use them!

Because the constants are part of the spec that will be documented generally and I am not sure it has enough value making them part of the public API in every SDK (nor do I think that we need to with the other environment variable names such as TEMPORAL_DEBUG). Clear environment variable names from spec I don't think need to also be exposed user-facing constants unlike other string literals which might.

contrib/envconfig/client_config.go

yuandrew · 2025-02-25T17:16:06Z

contrib/envconfig/client_config.go

+}
+
+// ToClientOptionsOptions are options for [ClientConfig.ToClientOptions] and [ClientConfigProfile.ToClientOptions].
+type ToClientOptionsOptions struct {


OptionsOptions feels a little awkward for a name, but can't think of a great alternative..

Agreed, I had brought this up internally but no good solution was provided. I don't expect users to use this part of the API much.

contrib/envconfig/client_config_load.go

tomwheeler

@cretz I updated my local copy of your Go SDK to commit #362043e and then tried this again.

Following the approach previously described, I tested this with mTLS and found that it now works as expected.

I then attempted to do the same with API Key authentication. I unset the previous four environment variables and then set the following:

TEMPORAL_API_KEY=<redacted>
TEMPORAL_NAMESPACE=tom-dev.a2dd6
TEMPORAL_ADDRESS=us-east-1.aws.api.temporal.io:7233

When I started the Worker, it failed:

$ go run worker/main.go
2025/02/25 13:53:31 INFO  No logger configured for temporal client. Created default one.
2025/02/25 13:53:32 Unable to create Temporal client. failed reaching server: rpc error: code = Unauthenticated desc = Jwt is expired
exit status 1

I then looked at the code and realized that perhaps I needed to run this:

$ export TEMPORAL_TLS=true

I then repeated the previous command and found that the Worker now started. I did the same in the other terminal before running the command to start the Workflow, which then completed successfully.

In the end, it seems to work fine, although I think we'd be wise to emphasize in the docs that you must set TEMPORAL_TLS=true when using API Key authentication with Temporal Cloud, because I think people might overlook it otherwise.

cretz · 2025-02-25T23:41:32Z

Jwt is expired

@tomwheeler - that is the error you got originally, I am not sure how TEMPORAL_TLS=true affects that. For env config, we are defaulting TLS to true if you set the API key (all of this is not documented yet because so much may change during this review process), so you should not have to set it manually. I will confirm this is the case. Also, feel free to contact via internal channels if that'd be easier.

EDIT: @tomwheeler - I cannot replicate. With the samples-go project checked out to temporalio/samples-go#392 and the SDK it references checked out to this PR, when I run this in one terminal (Windows, change set to export for nix):

set TEMPORAL_ADDRESS=us-west-2.aws.api.temporal.io:7233
set TEMPORAL_NAMESPACE=sdk-ci.a2dd6
set TEMPORAL_API_KEY=omitted
go run ./helloworld/worker

and in another terminal

set TEMPORAL_ADDRESS=us-west-2.aws.api.temporal.io:7233
set TEMPORAL_NAMESPACE=sdk-ci.a2dd6
set TEMPORAL_API_KEY=omitted
go run ./helloworld/starter

It works as expected. TEMPORAL_TLS is not needed.

Quinn-With-Two-Ns · 2025-03-12T07:02:46Z

contrib/envconfig/client_config.go

+//
+// WARNING: Environment configuration is currently experimental.
+type ClientConfigProfile struct {
+	Address   string


Don't we generally want all public structs documented?

Yes, though I am not sure it necessarily applies to all public struct fields. I can document it, though I will likely just say it applies to certain client option fields and not give all the details.

Documented struct fields

Quinn-With-Two-Ns · 2025-03-12T07:03:29Z

contrib/envconfig/client_config.go

+//
+// WARNING: Environment configuration is currently experimental.
+type ClientConfigTLS struct {
+	Disabled                bool


So the default will be TLS enabled?

Or are we treating nil different then the zero value?

So the default will be TLS enabled?

No, the default is "depends on other fields". If you give an API key the default is enabled, but if you don't the default is disabled (but you can make enabled with any TLS value including TLS itself being set). So this is effectively for those rare cases that have an API key but want to explicitly disable TLS (e.g. self-hosted using JWTs). This is what allows us to just accept an API key env var for cloud and assume TLS.

I was jsut confused here because the default of the bool will be false, so that could imply the default of TLS is enable

Unset/false here is "not disabled", not necessarily "enabled". I agree it can be confusing, but I think any alternative is more confusing. Open to other suggestions. We don't want to differentiate between false and unset though as they are the same thing.

Quinn-With-Two-Ns · 2025-03-12T07:10:48Z

contrib/envconfig/client_config_load.go

+}
+
+// LoadClientOptionsOptions are options for [LoadClientOptions].
+type LoadClientOptionsOptions struct {


I thought we said we didn't want to stutter on Options here, generally the SDK has used request instead of option when it hits this

I didn't think we had settled on this. I thought request was for actual API requests (i.e. RPC calls). But I can change ToClientOptionsOptions to ToClientOptionsRequest if you think that's better. It's a bit confusing that some are requests and some are options when they are really all options.

Bump here. Do you think that it's worth suffixing with Request even though these aren't really requests like you'd have on a client?

I am fine with it being Request even though it is not a client call. OK leaving it as Option as well since we are consistent in this module at least

Changed to OptionsRequest

cretz requested a review from a team as a code owner February 25, 2025 14:45

This was referenced Feb 25, 2025

💥 Support environment config [BREAKING CHANGES] temporalio/cli#764

Merged

Support environment config on a single sample temporalio/samples-go#392

Open

Support environment configuration

12d159d

Fixes temporalio#1815

cretz force-pushed the envconfig branch from c290f4e to 12d159d Compare February 25, 2025 14:54

cretz added 2 commits February 25, 2025 07:04

Merge branch 'master' into envconfig

678d714

go.mod tidy

362043e

Sushisource reviewed Feb 25, 2025

View reviewed changes

yuandrew reviewed Feb 25, 2025

View reviewed changes

tomwheeler self-requested a review February 25, 2025 20:10

tomwheeler reviewed Feb 25, 2025

View reviewed changes

Quinn-With-Two-Ns reviewed Mar 12, 2025

View reviewed changes

cretz added 3 commits March 26, 2025 13:03

Minor code comments

daaae5b

Minor PR suggestions

133f461

Merge branch 'master' into envconfig

ddaa30d

Sushisource approved these changes Mar 31, 2025

View reviewed changes

Quinn-With-Two-Ns approved these changes Mar 31, 2025

View reviewed changes

Merge branch 'master' into envconfig

91d9f9c

cretz merged commit 4c897cd into temporalio:master Apr 1, 2025
13 checks passed

cretz deleted the envconfig branch April 1, 2025 14:21

Support environment configuration #1849

Support environment configuration #1849

Uh oh!

Conversation

cretz commented Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What was changed

Checklist

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sushisource Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Mar 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tomwheeler left a comment

Choose a reason for hiding this comment

Uh oh!

cretz commented Feb 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cretz Mar 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cretz commented Feb 25, 2025 •

edited

Loading

cretz Feb 25, 2025 •

edited

Loading

cretz Feb 25, 2025 •

edited

Loading

cretz Feb 25, 2025 •

edited

Loading

Sushisource Mar 24, 2025 •

edited

Loading

cretz Mar 25, 2025 •

edited

Loading

cretz commented Feb 25, 2025 •

edited

Loading

cretz Mar 26, 2025 •

edited

Loading

cretz Mar 17, 2025 •

edited

Loading