get read-only common configs for custom application

[kamauz]

Situation:

• common configurations placed in a local yaml file

Goal:

• need to access the common configuration from the main function, in order to combine it with custom configs
  (es. cdc with database credentials passed from a common configuration)

Given the following snippet of code, I need to populate config.ServiceConfig.Database with service.config.Database , however service properties are not accessible

func main() {
	service, ok := pkg.NewAppServiceWithTargetType(serviceKey, &[]byte{})
	if !ok {
		service.LoggingClient().Errorf("App Service initialization failed for %s", serviceKey)
		os.Exit(-1)
	}

	lc := service.LoggingClient()

	config := config.ServiceConfig{}
        

        ////////////////////////////
        // <--- 
        // service.config.Database = service.config.Database   (config is a private property, not accessible)
        ////////////////////////////


	if err := service.LoadCustomConfig(&config, "CDC"); err != nil {
		lc.Errorf("LoadCustomConfig returned error: %s", err.Error())
		os.Exit(-1)
	}

	service.RegisterCustomTriggerFactory("cdc", func(config interfaces.TriggerConfig) (interfaces.Trigger, error) {
		return trigger.NewCDCTrigger(config), nil
	})
}

So I moved the focus on "LoadCustomConfig" function, but it would expect to read common configurations from a Consul.
Otherwise the private configuration is loaded as-is from the default path (res/configuration.yaml)
here's the code I'm referring to:

bootstrap/config/config.go

func (cp *Processor) LoadCustomConfigSection(updatableConfig interfaces.UpdatableConfig, sectionName string) error {
	if cp.envVars == nil {
		cp.envVars = environment.NewVariables(cp.lc)
	}

	configClient := container.ConfigClientFrom(cp.dic.Get)

        // no Configuration provider specified
	if configClient == nil {

/////////////// START /////////////////////

		cp.lc.Info("Skipping use of Configuration Provider for custom configuration: Provider not available")
		filePath := GetConfigFileLocation(cp.lc, cp.flags)
		configMap, err := cp.loadConfigYamlFromFile(filePath)
		if err != nil {
			return err
		}

		err = utils.ConvertFromMap(configMap, updatableConfig)
		if err != nil {
			return fmt.Errorf("failed to convert custom configuration into service's configuration: %v", err)
		}

/////////////// END /////////////////////

	} else {
		cp.lc.Infof("Checking if custom configuration ('%s') exists in Configuration Provider", sectionName)

		exists, err := configClient.HasSubConfiguration(sectionName)
............

[cloudxxx8]

If you don't want to use Consul to manage your config, you can disable the config provider via the environment variable
https://docs.edgexfoundry.org/3.1/microservices/configuration/CommonEnvironmentVariables/#edgex_config_provider
EDGEX_CONFIG_PROVIDER: "none"

Then, you can manage your CustomConfig in your file system.
However, the config should be read-only at runtime, and the credentials should not be passed from common config.
If you don't persist your credentials in the secret store, you can use SecretProvider() bootstrapInterfaces.SecretProvider to retrieve the credentials from the insecure section.

app-functions-sdk-go/pkg/interfaces/service.go

Line 149 in d5728ff

SecretProvider() bootstrapInterfaces.SecretProvider

Config example:
https://github.com/edgexfoundry/app-service-configurable/blob/99abc15c93f70f8386f8be99e96e68a0293bb7bf/res/mqtt-export/configuration.yaml#L70
https://docs.edgexfoundry.org/3.1/design/adr/014-Secret-Provider-For-All/

[sbaeurle]

Quick follow up regarding the question above since I'm facing a similar issue with my custom application service.

I want to store some messages as metadata in the built-in redis database. However the DB Store of the SDK is only available in the internal package and not exported.
Thus I created my own custom redis client that still relies on the configuration and credentials provided by the config provider.

// Create new metadata handler to be used in the functions pipeline
func NewMetadataHandler(config config.Database, credentials config.Credentials) (*MetadataHandler, error)

Proposed usage in main.go

metadataHandler, err := functions.NewMetadataHandler(app.config.Database, app.config.Credentials)
if err != nil {
	app.lc.Errorf("failed load custom configuration: %s", err.Error())
	return -1
}

 app.service.SetDefaultFunctionsPipeline(metadataHandler.HandleMetadata)

Is it possible to either export the DB Store for easy access in custom application functions or at least access the rest of the configuration using the app functions SDK?
Or am i potentially missing some other way to achieve this (maybe using some different SDK)?

[cloudxxx8]

DB Store in App Service is designed for store and forward only, so it should not be exposed.
The credentials can be retrieved from the SecretProvider() as my previous post. It should be enough for creating your own database connection.

To expose the whole service config, it requires some more effort and we are not able to make it by the current release. You may open an issue with the detailed design and discuss with the Technical Working Group about how to implement it.