|
| 1 | +# Configuration Data Model |
| 2 | + |
| 3 | +**Status**: [Development](../document-status.md) |
| 4 | + |
| 5 | +<!-- toc --> |
| 6 | + |
| 7 | +- [Overview](#overview) |
| 8 | + * [Stability definition](#stability-definition) |
| 9 | + * [File-based configuration model](#file-based-configuration-model) |
| 10 | + + [YAML file format](#yaml-file-format) |
| 11 | + + [Environment variable substitution](#environment-variable-substitution) |
| 12 | + |
| 13 | +<!-- tocstop --> |
| 14 | + |
| 15 | +## Overview |
| 16 | + |
| 17 | +The OpenTelemetry configuration data model is part of |
| 18 | +the [declarative configuration interface](./README.md#declarative-configuration). |
| 19 | + |
| 20 | +The data model defines data structures which allow users to specify an intended |
| 21 | +configuration of OpenTelemetry SDK components and instrumentation. |
| 22 | + |
| 23 | +The data model is defined |
| 24 | +in [opentelemetry-configuration](https://github.com/open-telemetry/opentelemetry-configuration) |
| 25 | +using [JSON Schema](https://json-schema.org/). |
| 26 | + |
| 27 | +The data model itself is an abstraction with multiple built-in representations: |
| 28 | + |
| 29 | +* [File-based configuration model](#file-based-configuration-model) |
| 30 | +* [SDK in-memory configuration model](./sdk.md#in-memory-configuration-model) |
| 31 | + |
| 32 | +### Stability definition |
| 33 | + |
| 34 | +TODO: define stability guarantees and backwards compatibility |
| 35 | + |
| 36 | +### File-based configuration model |
| 37 | + |
| 38 | +A configuration file is a serialized file-based representation of |
| 39 | +the configuration data model. |
| 40 | + |
| 41 | +Configuration files SHOULD use one the following serialization formats: |
| 42 | + |
| 43 | +* [YAML file format](#yaml-file-format) |
| 44 | + |
| 45 | +#### YAML file format |
| 46 | + |
| 47 | +[YAML](https://yaml.org/spec/1.2.2/) configuration files SHOULD follow YAML spec |
| 48 | +revision >= 1.2. |
| 49 | + |
| 50 | +YAML configuration files SHOULD be parsed using [v1.2 YAML core schema](https://yaml.org/spec/1.2.2/#103-core-schema). |
| 51 | + |
| 52 | +YAML configuration files MUST use file extensions `.yaml` or `.yml`. |
| 53 | + |
| 54 | +#### Environment variable substitution |
| 55 | + |
| 56 | +Configuration files support environment variables substitution for references |
| 57 | +which match the following PCRE2 regular expression: |
| 58 | + |
| 59 | +```regexp |
| 60 | +\$\{(?:env:)?(?<ENV_NAME>[a-zA-Z_][a-zA-Z0-9_]*)(:-(?<DEFAULT_VALUE>[^\n]*))?\} |
| 61 | +``` |
| 62 | + |
| 63 | +The `ENV_NAME` MUST start with an alphabetic or `_` character, and is followed |
| 64 | +by 0 or more alphanumeric or `_` characters. |
| 65 | + |
| 66 | +For example, `${API_KEY}` and `${env:API_KEY}` are valid, while `${1API_KEY}` |
| 67 | +and `${API_$KEY}` are invalid. |
| 68 | + |
| 69 | +Environment variable substitution MUST only apply to scalar values. Mapping keys |
| 70 | +are not candidates for substitution. |
| 71 | + |
| 72 | +The `DEFAULT_VALUE` is an optional fallback value which is substituted |
| 73 | +if `ENV_NAME` is null, empty, or undefined. `DEFAULT_VALUE` consists of 0 or |
| 74 | +more non line break characters (i.e. any character except `\n`). If a referenced |
| 75 | +environment variable is not defined and does not have a `DEFAULT_VALUE`, it MUST |
| 76 | +be replaced with an empty value. |
| 77 | + |
| 78 | +When parsing a configuration file that contains a reference not matching |
| 79 | +the references regular expression but does match the following PCRE2 |
| 80 | +regular expression, the parser MUST return an empty result (no partial |
| 81 | +results are allowed) and an error describing the parse failure to the user. |
| 82 | + |
| 83 | +```regexp |
| 84 | +\$\{(?<INVALID_IDENTIFIER>[^}]+)\} |
| 85 | +``` |
| 86 | + |
| 87 | +Node types MUST be interpreted after environment variable substitution takes |
| 88 | +place. This ensures the environment string representation of boolean, integer, |
| 89 | +or floating point fields can be properly converted to expected types. |
| 90 | + |
| 91 | +It MUST NOT be possible to inject YAML structures by environment variables. For |
| 92 | +example, see references to `INVALID_MAP_VALUE` environment variable below. |
| 93 | + |
| 94 | +It MUST NOT be possible to inject environment variable by environment variables. |
| 95 | +For example, see references to `DO_NOT_REPLACE_ME` environment variable below. |
| 96 | + |
| 97 | +For example, consider the following environment variables, |
| 98 | +and [YAML](#yaml-file-format) configuration file: |
| 99 | + |
| 100 | +```shell |
| 101 | +export STRING_VALUE="value" |
| 102 | +export BOOL_VALUE="true" |
| 103 | +export INT_VALUE="1" |
| 104 | +export FLOAT_VALUE="1.1" |
| 105 | +export HEX_VALUE="0xdeadbeef" # A valid integer value written in hexadecimal |
| 106 | +export INVALID_MAP_VALUE="value\nkey:value" # An invalid attempt to inject a map key into the YAML |
| 107 | +export DO_NOT_REPLACE_ME="Never use this value" # An unused environment variable |
| 108 | +export REPLACE_ME='${DO_NOT_REPLACE_ME}' # A valid replacement text, used verbatim, not replaced with "Never use this value" |
| 109 | +``` |
| 110 | + |
| 111 | +```yaml |
| 112 | +string_key: ${STRING_VALUE} # Valid reference to STRING_VALUE |
| 113 | +env_string_key: ${env:STRING_VALUE} # Valid reference to STRING_VALUE |
| 114 | +other_string_key: "${STRING_VALUE}" # Valid reference to STRING_VALUE inside double quotes |
| 115 | +another_string_key: "${BOOL_VALUE}" # Valid reference to BOOL_VALUE inside double quotes |
| 116 | +string_key_with_quoted_hex_value: "${HEX_VALUE}" # Valid reference to HEX_VALUE inside double quotes |
| 117 | +yet_another_string_key: ${INVALID_MAP_VALUE} # Valid reference to INVALID_MAP_VALUE, but YAML structure from INVALID_MAP_VALUE MUST NOT be injected |
| 118 | +bool_key: ${BOOL_VALUE} # Valid reference to BOOL_VALUE |
| 119 | +int_key: ${INT_VALUE} # Valid reference to INT_VALUE |
| 120 | +int_key_with_unquoted_hex_value: ${HEX_VALUE} # Valid reference to HEX_VALUE without quotes |
| 121 | +float_key: ${FLOAT_VALUE} # Valid reference to FLOAT_VALUE |
| 122 | +combo_string_key: foo ${STRING_VALUE} ${FLOAT_VALUE} # Valid reference to STRING_VALUE and FLOAT_VALUE |
| 123 | +string_key_with_default: ${UNDEFINED_KEY:-fallback} # UNDEFINED_KEY is not defined but a default value is included |
| 124 | +undefined_key: ${UNDEFINED_KEY} # Invalid reference, UNDEFINED_KEY is not defined and is replaced with "" |
| 125 | +${STRING_VALUE}: value # Invalid reference, substitution is not valid in mapping keys and reference is ignored |
| 126 | +recursive_key: ${REPLACE_ME} # Valid reference to REPLACE_ME |
| 127 | +# invalid_identifier_key: ${STRING_VALUE:?error} # If uncommented, this is an invalid identifier, it would fail to parse |
| 128 | +``` |
| 129 | + |
| 130 | +Environment variable substitution results in the following YAML: |
| 131 | + |
| 132 | +```yaml |
| 133 | +string_key: value # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 134 | +env_string_key: value # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 135 | +other_string_key: "value" # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 136 | +another_string_key: "true" # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 137 | +string_key_with_quoted_hex_value: "0xdeadbeef" # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 138 | +yet_another_string_key: "value\nkey:value" # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 139 | +bool_key: true # Interpreted as type bool, tag URI tag:yaml.org,2002:bool |
| 140 | +int_key: 1 # Interpreted as type int, tag URI tag:yaml.org,2002:int |
| 141 | +int_key_with_unquoted_hex_value: 3735928559 # Interpreted as type int, tag URI tag:yaml.org,2002:int |
| 142 | +float_key: 1.1 # Interpreted as type float, tag URI tag:yaml.org,2002:float |
| 143 | +combo_string_key: foo value 1.1 # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 144 | +string_key_with_default: fallback # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 145 | +undefined_key: # Interpreted as type null, tag URI tag:yaml.org,2002:null |
| 146 | +${STRING_VALUE}: value # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 147 | +recursive_key: ${DO_NOT_REPLACE_ME} # Interpreted as type string, tag URI tag:yaml.org,2002:str |
| 148 | +``` |
0 commit comments