Skip to content

feat(config): add HARPER_CONFIG as the recommended way to set config from the environment #1238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kriszyp merged 2 commits into from
Jun 11, 2026
Merged

feat(config): add HARPER_CONFIG as the recommended way to set config from the environment #1238

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
016a8e6
feat(config): add HARPER_CONFIG as the recommended mergeable config e…
heskew Jun 10, 2026
826a3a6
fix(config): run env-config cleanup when all vars removed after a pri…
heskew Jun 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions config/configUtils.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ function createConfigFile(args, skipFsValidation = false) {

if (schemasArgs) setSchemasConfig(configDoc, schemasArgs);

// Apply HARPER_DEFAULT_CONFIG and HARPER_SET_CONFIG environment variables BEFORE validation
// Apply HARPER_DEFAULT_CONFIG, HARPER_CONFIG and HARPER_SET_CONFIG environment variables BEFORE validation
// This allows runtime env vars to resolve port conflicts before validation
// Must be called AFTER rootPath is set in configDoc
// Mutates configDoc in place
Expand Down Expand Up @@ -352,7 +352,7 @@ function initConfig(force = false) {

checkForUpdatedConfig(configDoc, configFilePath);

// Apply HARPER_DEFAULT_CONFIG and HARPER_SET_CONFIG environment variables
// Apply HARPER_DEFAULT_CONFIG, HARPER_CONFIG and HARPER_SET_CONFIG environment variables
applyRuntimeEnvVarConfig(configDoc, configFilePath);

// Validates config doc and if required sets default values for some parameters.
Expand Down Expand Up @@ -848,15 +848,16 @@ function parseYamlDoc(filePath) {
}

/**
* Apply HARPER_DEFAULT_CONFIG and HARPER_SET_CONFIG environment variables at runtime
* Apply HARPER_DEFAULT_CONFIG, HARPER_CONFIG and HARPER_SET_CONFIG environment variables at runtime
*
* This function performs the following:
* 1. Loads configuration state to track sources
* 2. Detects user edits (drift) to protect them from HARPER_DEFAULT_CONFIG
* 3. Applies HARPER_DEFAULT_CONFIG (respects user edits)
* 4. Applies HARPER_SET_CONFIG (overrides everything)
* 5. Handles deletions when keys removed from env vars
* 6. Saves updated state and persists changes to config file (if configFilePath provided)
* 4. Applies HARPER_CONFIG (merge layer: reasserts its keys, yields only to HARPER_SET_CONFIG)
* 5. Applies HARPER_SET_CONFIG (overrides everything)
* 6. Handles deletions when keys removed from env vars
* 7. Saves updated state and persists changes to config file (if configFilePath provided)
*
* NOTE: This function performs multiple conversions (YAML → JSON → YAML) which is not
* efficient but provides clear separation of concerns. The conversions are necessary
Expand All @@ -869,10 +870,11 @@ function parseYamlDoc(filePath) {
*/
function applyRuntimeEnvVarConfig(configDoc, configFilePath, options = {}) {
const defaultEnvValue = process.env.HARPER_DEFAULT_CONFIG;
const configEnvValue = process.env.HARPER_CONFIG;
const setEnvValue = process.env.HARPER_SET_CONFIG;

// No env vars set, skip entirely (zero overhead)
if (!defaultEnvValue && !setEnvValue) return;
if (!defaultEnvValue && !configEnvValue && !setEnvValue) return;
Comment thread
heskew marked this conversation as resolved.
Outdated

const { applyRuntimeEnvConfig } = require('./harperConfigEnvVars.ts');

Expand Down
78 changes: 54 additions & 24 deletions config/harperConfigEnvVars.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,20 @@
/**
* HARPER_DEFAULT_CONFIG and HARPER_SET_CONFIG environment variable support
* HARPER_CONFIG, HARPER_DEFAULT_CONFIG and HARPER_SET_CONFIG environment variable support
*
* This module provides utilities for applying configuration from environment variables
* to Harper's configuration system with source tracking and drift detection.
*
* The three variables form a precedence ladder (later wins):
* HARPER_DEFAULT_CONFIG < config file / user edits < HARPER_CONFIG < HARPER_SET_CONFIG
*
* - HARPER_CONFIG (recommended): merge — sets exactly the keys it names, reasserting
* them on every boot (a manual edit to a named key is overwritten on restart), and
* yields only to HARPER_SET_CONFIG. Individual HARPER_* env vars still win over it
* for the keys they name (arg filtering remains SET-only).
* - HARPER_DEFAULT_CONFIG: defaults — yields to the config file, individual env vars,
* and user edits.
* - HARPER_SET_CONFIG: force — overrides everything and locks against drift.
*
* Features:
* - Install-time and runtime configuration from env vars
* - Source tracking (which env var set each config value)
Expand Down Expand Up @@ -31,7 +42,7 @@ function getLogger(): Logger {

// Type definitions
type ConfigObject = Record<string, any>;
type ConfigSource = 'HARPER_DEFAULT_CONFIG' | 'HARPER_SET_CONFIG' | 'user' | 'default';
type ConfigSource = 'HARPER_DEFAULT_CONFIG' | 'HARPER_CONFIG' | 'HARPER_SET_CONFIG' | 'user' | 'default';

/**
* Configuration state tracking structure
Expand Down Expand Up @@ -69,6 +80,7 @@ interface ConfigState {
snapshots: {
// Snapshots of what each env var currently specifies (for detecting changes)
HARPER_DEFAULT_CONFIG?: { hash: string; config: ConfigObject };
HARPER_CONFIG?: { hash: string; config: ConfigObject };
HARPER_SET_CONFIG?: { hash: string; config: ConfigObject };
};
}
Expand Down Expand Up @@ -534,9 +546,11 @@ function handleDeletions(
for (const path of deletedPaths) {
// Only handle if this path was set by this source
if (state.sources[path] === sourceName) {
// For both HARPER_DEFAULT_CONFIG and HARPER_SET_CONFIG, restore original value instead of deleting
// For all config env vars, restore original value instead of deleting
if (
(sourceName === 'HARPER_DEFAULT_CONFIG' || sourceName === 'HARPER_SET_CONFIG') &&
(sourceName === 'HARPER_DEFAULT_CONFIG' ||
sourceName === 'HARPER_CONFIG' ||
sourceName === 'HARPER_SET_CONFIG') &&
path in state.originalValues
) {
setNestedValue(fileConfig, path, state.originalValues[path]);
Expand Down Expand Up @@ -608,6 +622,14 @@ function processEnvVar(
respectSources: [],
storeOriginals: true,
});
} else if (sourceName === 'HARPER_CONFIG') {
// HARPER_CONFIG merges: it sets exactly the keys it names and reasserts them on
// every boot (winning over the config file, user edits, and DEFAULT), yielding
// only to HARPER_SET_CONFIG. Same behavior at install and runtime.
applyConfigLayer(fileConfig, state, parsedConfig, sourceName, {
respectSources: ['HARPER_SET_CONFIG'],
storeOriginals: true,
});
} else if (sourceName === 'HARPER_DEFAULT_CONFIG') {
// DEFAULT_CONFIG behavior depends on install vs runtime
if (options.isInstall) {
Expand Down Expand Up @@ -675,8 +697,8 @@ function cleanupRemovedEnvVar(

const logger = getLogger();

// For both HARPER_DEFAULT_CONFIG and HARPER_SET_CONFIG, restore original values
if (sourceName === 'HARPER_DEFAULT_CONFIG' || sourceName === 'HARPER_SET_CONFIG') {
// For all config env vars, restore original values
if (sourceName === 'HARPER_DEFAULT_CONFIG' || sourceName === 'HARPER_CONFIG' || sourceName === 'HARPER_SET_CONFIG') {
const pathsToCleanup = Object.keys(state.sources).filter((path) => state.sources[path] === sourceName);
for (const path of pathsToCleanup) {
if (path in state.originalValues) {
Expand All @@ -699,14 +721,15 @@ function cleanupRemovedEnvVar(
}

/**
* Compose a merged config from HARPER_DEFAULT_CONFIG and HARPER_SET_CONFIG
* layered with an optional base. Later layers win:
* HARPER_DEFAULT_CONFIG < base < HARPER_SET_CONFIG
* Compose a merged config from HARPER_DEFAULT_CONFIG, HARPER_CONFIG and
* HARPER_SET_CONFIG layered with an optional base. Later layers win:
* HARPER_DEFAULT_CONFIG < base < HARPER_CONFIG < HARPER_SET_CONFIG
*
* HARPER_DEFAULT_CONFIG provides scaffolding defaults, the base (e.g., the
* user's existing config file) is layered on top, and HARPER_SET_CONFIG
* force-overrides everything. This matches the precedence applied by the
* runtime pipeline in applyRuntimeEnvConfig.
* user's existing config file) is layered on top, HARPER_CONFIG merges its
* keys over that, and HARPER_SET_CONFIG force-overrides everything. This
* matches the precedence applied by the runtime pipeline in
* applyRuntimeEnvConfig.
*
* Unlike applyRuntimeEnvConfig, this does NOT read or write the config state
* file and does NOT track sources — it returns a fresh object. Use when you
Expand All @@ -718,6 +741,7 @@ export function composeConfigFromEnv(base: ConfigObject = {}): ConfigObject {
const layers: (ConfigObject | null)[] = [
parseConfigEnvVar(process.env.HARPER_DEFAULT_CONFIG, 'HARPER_DEFAULT_CONFIG'),
cloneDeep(base),
parseConfigEnvVar(process.env.HARPER_CONFIG, 'HARPER_CONFIG'),
parseConfigEnvVar(process.env.HARPER_SET_CONFIG, 'HARPER_SET_CONFIG'),
];

Expand All @@ -733,22 +757,23 @@ export function composeConfigFromEnv(base: ConfigObject = {}): ConfigObject {
}

/**
* Apply HARPER_DEFAULT_CONFIG and HARPER_SET_CONFIG
* Can be used for both install-time and runtime
* Apply HARPER_DEFAULT_CONFIG, HARPER_CONFIG and HARPER_SET_CONFIG (in that order —
* later wins). Can be used for both install-time and runtime.
*/
export function applyRuntimeEnvConfig(
fileConfig: ConfigObject,
rootPath: string,
options: { isInstall?: boolean } = {}
): ConfigObject {
const defaultEnvValue = process.env.HARPER_DEFAULT_CONFIG;
const configEnvValue = process.env.HARPER_CONFIG;
const setEnvValue = process.env.HARPER_SET_CONFIG;

// Load existing state
const state = loadConfigState(rootPath);

// No env vars set and no previous state, nothing to do
if (!defaultEnvValue && !setEnvValue && Object.keys(state.snapshots).length === 0) {
if (!defaultEnvValue && !configEnvValue && !setEnvValue && Object.keys(state.snapshots).length === 0) {
return fileConfig;
}

Expand All @@ -760,22 +785,27 @@ export function applyRuntimeEnvConfig(
}
}

// Process HARPER_DEFAULT_CONFIG
processEnvVar(fileConfig, state, 'HARPER_DEFAULT_CONFIG', 'HARPER_DEFAULT_CONFIG', options);

// Clean up if HARPER_DEFAULT_CONFIG was removed
// Clean up any env var that was removed BEFORE applying the remaining ones. A removed
// var restores its paths to their stored originals and clears ownership; doing this
// first means a path a higher-precedence var is releasing is already un-sourced when a
// lower-precedence var (e.g. HARPER_CONFIG) runs, so that var reclaims it the same boot
// instead of leaving it at the file value for one boot.
if (!defaultEnvValue) {
cleanupRemovedEnvVar(fileConfig, state, 'HARPER_DEFAULT_CONFIG', 'HARPER_DEFAULT_CONFIG');
}

// Process HARPER_SET_CONFIG (always overrides everything)
processEnvVar(fileConfig, state, 'HARPER_SET_CONFIG', 'HARPER_SET_CONFIG', options);

// Clean up if HARPER_SET_CONFIG was removed
if (!configEnvValue) {
cleanupRemovedEnvVar(fileConfig, state, 'HARPER_CONFIG', 'HARPER_CONFIG');
}
if (!setEnvValue) {
cleanupRemovedEnvVar(fileConfig, state, 'HARPER_SET_CONFIG', 'HARPER_SET_CONFIG');
}

// Apply present vars in precedence order (later wins):
// HARPER_DEFAULT_CONFIG < config file / user edits < HARPER_CONFIG < HARPER_SET_CONFIG
processEnvVar(fileConfig, state, 'HARPER_DEFAULT_CONFIG', 'HARPER_DEFAULT_CONFIG', options);
processEnvVar(fileConfig, state, 'HARPER_CONFIG', 'HARPER_CONFIG', options);
processEnvVar(fileConfig, state, 'HARPER_SET_CONFIG', 'HARPER_SET_CONFIG', options);

// Save updated state
saveConfigState(rootPath, state);

Expand Down
Loading
Loading