Support dropin config files (within `.d` directories) #849

thorsten-klein · 2025-09-27T05:02:46Z

Successor of #847

Why?

In a big company there are often shared configs for build system and tooling (e.g., west and its configs).
Those generic configs must be applied first and contain some default or fallback values, if the user does not specify any other value in his user config. Remember, that users must always be able to overwrite values from those generic config files within their config.

Solution

The user can enable dropin configs. If enabled with option config.dropins = true, all .conf and .ini files that are placed inside according dropin folder (according <config>.d folder next to the config) are applied right before the config.

The dropin config folders are:

./west/config.d/* for local dropin configs
~/.westconfig.d/* for global dropin config
etc/westconfig.d/*/ /usr/local/etc/westconfig.d/* / %PROGRAMDATA%\west\config.d/* for system dropin config

All dropin configs are loaded in alphabetical order before the config file itself is applied.
If the same option/key/value is defined in multiple dropin configs, the last one loaded wins (i.e., later file (e.g. 99.ini) overrides earlier ones (e.g. 01.conf)).
In the end, the config file itself is applied, so if a option's value is defined in the config it always overrides values from dropin configs.

Note: The dropin config mechanism respects the env variables (WEST_CONFIG_LOCAL/WEST_CONFIG_SYSTEM/WEST_CONFIG_GLOBAL), so that the dropin config folder becomes <WEST_CONFIG_xxx>.d.

Note: The dropin config feature must be enabled per config type (local, global, system) by setting option config.dropins = true in the according config. This is important to ensure security. For example: If the option is enabled in global config, only global dropin configs are used (not system or local ones). By this it is avoided that arbitrary dropin configs from any local workspaces are considered. As a result this also means, that dropin configs are never used if the according config file does not exist (since it would have to actively enable this option).

A new west config [--local|--global|--system] --print-path command is supported. This command prints the path of the currently considered config file.
Note: If no local config can be determined, the --local command does not output anything (it does NOT fail).

A new west config [--local|--global|--system] --list-paths command is added, that prints a list of config files that are currently used by west commands in this workspace (existing config files and dropin config).

codecov · 2025-09-27T05:11:15Z

Codecov Report

❌ Patch coverage is 95.87629% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.88%. Comparing base (a3af732) to head (b9ee7fc).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
src/west/configuration.py	95.00%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #849      +/-   ##
==========================================
+ Coverage   84.52%   84.88%   +0.36%     
==========================================
  Files          11       11              
  Lines        3366     3428      +62     
==========================================
+ Hits         2845     2910      +65     
+ Misses        521      518       -3

Files with missing lines	Coverage Δ
src/west/app/config.py	`94.44% <100.00%> (+0.82%)`	⬆️
src/west/configuration.py	`86.68% <95.00%> (+1.76%)`	⬆️

... and 2 files with indirect coverage changes

pdgendt

Needs a rebase, Python 3.9 support has been removed from main.

src/west/configuration.py

pdgendt

I know this PR was created based on a suggestion from #847 (comment), but it does add some complexity to the configuration system.
Besides that, testing now has a lot more edge-cases.

At the very least we require some more documentation bits. But I'm not convinced yet that this addition is in the spirit of:

Conservative about features: no features will be accepted without strong and compelling motivation.

Can you provide some insight into where you "really" need this kind of flexibility?

pdgendt · 2025-09-29T11:27:39Z

src/west/configuration.py

+            for conf in sorted(dropins_dir.iterdir()):
+                config.cp.read(dropins_dir / conf, encoding='utf-8')
+            # finally overwrite with values from higher-prior actual config
+            config.cp.read(path, encoding='utf-8')


So you need to read the "main" config file twice to make sure it has precedence over the config.d/ files. That feels a bit hacky.

Would it be okay if I add some argument to the constructor of the config which allows to create the instance without directly reading the file?

config.cp.read actually allows passing multiple filenames, why not pass multiple paths to the __init__ function?

There is a check at later point that the number of read configs is exactly 1. I can remove this check and rework the for loop and instead pass the list of multiple configs.

Nevertheless: what should be the path attribute of the class in case of multiple configs? This is why I thought that requiring the actual config is the best idea.

IMO if you have used multiple paths to load the configuration, it could be the argument for the constructor. It would actually allow the --print-path to print all used paths. Because now this is a hidden feature.

Good point. I like the idea to use the path of the config as argument for the constructor and within the constructor all according dropin config files are searched and their paths are stored in self.dropins .

Before I make those adaptions, maybe we can clarify first, if you will accept this feature. See my other comment in the conversation with some explanation what we want to do.

Let's use this conversation for clarification of open questions in order to not spam the issue conversation.

There is a check at later point that the number of read configs is exactly 1. I can remove this check and rework the for loop and instead pass the list of multiple configs.

Yes please. Also, I think it's better to just add all the code to the __init__ constructor, not to some convenience wrapper.

Nevertheless: what should be the path attribute of the class in case of multiple configs? This is why I thought that requiring the actual config is the best idea.

path should be the non-.d/ file. If it does not exist, then maybe it should be None? Not sure about that. Just like before, writes to a config file that does not exist yet should create that file, and deletions or appending in a non-existing main file should fail. If there was no test coverage for these then it should be added because the new "drop-ins" are going to make the corresponding logic a bit more complicated.

Writing to .d/ files should not be possible through the CLI, way too complicated. This must be documented.

Before I make those adaptions, maybe we can clarify first, if you will accept this feature. See my other comment in the conversation with some explanation what we want to do.

For me, it mostly depends on the complexity of the feature and how intuitive it's for users. For example,

Writing to .d/ files should not be possible through the CLI, way too complicated. This must be documented.

What about deletions? An option can come from any of those files, but they can't be deleted if not coming from the main config file? This feels somewhat counter-intuitive.

I'd like some other opinions from @carlescufi or @mbolivar too.

There is a check at later point that the number of read configs is exactly 1. I can remove this check and rework the for loop and instead pass the list of multiple configs.

Yes please. Also, I think it's better to just add all the code to the __init__ constructor, not to some convenience wrapper.

I agree here.

What about deletions? An option can come from any of those files, but they can't be deleted

Yes. In this case the user must set the value to the default value explicitly. I think this is very transparent as the user can track in the config files the value of each option. If we allow deleting an option in the config, it might be confusing for users where the different value comes from.

thorsten-klein · 2025-09-29T12:12:26Z

Can you provide some insight into where you "really" need this kind of flexibility?

In our company, only a small group of people maintains the build system and tooling and configs (e.g., west).
Their goal is to set up the development environment correctly and shield everyone else from pitfalls.
Similarly to this, also customer repositories may define configurations (on top of the generic configs) that should be shared across their product developers to ensure consistency.
This leads to the need for maintaining (generic) configs and a mechanism to overlay configs.
In the end, maybe something like a "repository config level" is missing for us.

I hope, that it is a common enough use case for others, so that west itself should support it.
(Especially, since the existence of other west config commands suggests that users already wanted to set options via the command line.)

My first idea was adding a command like west config --update.
It would allow us to ensure required config values are set. This could be implemented in only few lines (see #847).
Without such a mechanism, we would have to write our own tool for config-patching or a script which uses currently supported commands to set each option separately.

An alternative would be to support a dropin directory like .west/config.d directory.
This would allow us to ship shared configs and place them before the build in .west/config.d (or symlink the files/folder)
Users could still override values in .west/config, but in most cases that file would remain empty because all necessary options are maintained in the generic .west/config.d..

The important thing is: the generic configs may change anytime. In this case the new config values should be automatically applied if the users are using the default configs. If they have set some different value (on their own risk), their value shall be used. This scenario is perfectly fitting to the drop-in mechanism.

I hope you understand roughly what we do.

src/west/configuration.py

src/west/app/config.py

marc-hb · 2025-09-30T01:30:26Z

src/west/app/config.py


+    def print_path(self, args):
+        config_path = self.config.get_path(args.configfile or LOCAL)
+        if config_path:


How could this be None or empty? If it were, something went seriously wrong and it would be better to crash.

It can be None in case of local configs if there is no .west directory, so if west is run outside any workspace.
This makes sense for this command, e.g. if the user just opens a console in his home directory and runs west config --print-paths to see which configs are expected.

Would you expect that it fails in this case, or shall it exit 0 and print no path for the local config (as it is now)?

If the user requests the local path while outside any workspace, I would just let the tool crash. This is an obvious misuse of the tool, no need to test for that in the code.

I found a raise MalformedConfig('local configuration file not found') somewhere else, that's another option.

In any case I would never exit 0, the tool should not succeed.

PS: please do not reply with a question and then hide the thread as "resolved" :-D

src/west/configuration.py

marc-hb · 2025-09-30T02:23:37Z

src/west/configuration.py

+            for conf in sorted(dropins_dir.iterdir()):
+                config.cp.read(dropins_dir / conf, encoding='utf-8')
+            # finally overwrite with values from higher-prior actual config
+            config.cp.read(path, encoding='utf-8')


There is a check at later point that the number of read configs is exactly 1. I can remove this check and rework the for loop and instead pass the list of multiple configs.

Yes please. Also, I think it's better to just add all the code to the __init__ constructor, not to some convenience wrapper.

Nevertheless: what should be the path attribute of the class in case of multiple configs? This is why I thought that requiring the actual config is the best idea.

path should be the non-.d/ file. If it does not exist, then maybe it should be None? Not sure about that. Just like before, writes to a config file that does not exist yet should create that file, and deletions or appending in a non-existing main file should fail. If there was no test coverage for these then it should be added because the new "drop-ins" are going to make the corresponding logic a bit more complicated.

Writing to .d/ files should not be possible through the CLI, way too complicated. This must be documented.

pdgendt · 2025-09-30T06:46:52Z

An alternative would be to support a dropin directory like .west/config.d directory. This would allow us to ship shared configs and place them before the build in .west/config.d (or symlink the files/folder) Users could still override values in .west/config, but in most cases that file would remain empty because all necessary options are maintained in the generic .west/config.d..

The important thing is: the generic configs may change anytime. In this case the new config values should be automatically applied if the users are using the default configs. If they have set some different value (on their own risk), their value shall be used. This scenario is perfectly fitting to the drop-in mechanism.

I hope you understand roughly what we do.

Yes, I understand you want something like:

$ tree -a
.
├── .west
│   ├── config
│   └── config.d
│       └── 01-generic.conf -> ../../app/shared/01-generic.conf
├── app
│   └── shared
│       └── 01-generic.conf
└── zephyr

And get configuration updates when pulling the latest version of app. And I agree this can be powerful feature, but the edge cases need to be well defined.

mbolivar · 2025-09-30T14:57:12Z

Few initial thoughts:

Security

And I agree this can be powerful feature, but the edge cases need to be well defined.

Let's consider the security impact as well please.

I think @pdgendt 's example is a place where I can see how this is different than having an existing global or system config, but it makes me worried. We can assume that a ".d" directory in a home directory and anything in the system locations are controlled well by the user, but the workspace itself is an adversarial environment in my opinion -- that's why we can turn off extensions and why we do not parse extension command files unless the user specifically asks for one.

On Linux, I see no mention of XDG_CONFIG_HOME and I think we need to keep up with this standard for where these go.

marc-hb · 2025-10-01T00:39:58Z

but the workspace itself is an adversarial environment in my opinion

Is the 9f4dcf5 check not enough? If not, can you elaborate on the "threat model" you have in mind?

On Linux, I see no mention of XDG_CONFIG_HOME and I think we need to keep up with this standard for where these go.

XDG_CONFIG_HOME sounds like a different feature than this. Are you saying it should come first?

pdgendt · 2025-10-01T08:28:45Z

Before I make those adaptions, maybe we can clarify first, if you will accept this feature. See my other comment in the conversation with some explanation what we want to do.

@thorsten-klein The more I think about this, the more I dislike how this creates hidden paths for configuration bits. I was more in favor if updating/importing the configuration as was proposed with #847, which IMO has a clear intent and is a one-off action.
It does require more effort for users to keep up-to-date when updates are done to some shared files, but at least the user has to make a conscious decision.

One last alternative we did not bring up yet: we take inspiration from git, and git too has a way to extend configuration files, by adding include sections. Maybe we can use the same principle here? I am just spitballing here, but it would support your use-case, while maybe keeping things "simpler"?

thorsten-klein · 2025-10-01T09:33:11Z

It does require more effort for users to keep up-to-date when updates are done to some shared files, but at least the user has to make a conscious decision.

Exactly. That’s the main drawback with the approach from #847: we cannot easily tell whether a config was intentionally changed by the user, or if it’s simply outdated (or unintentionally modified). Therefore I prefer the dropin solution nowadays.

git too has a way to extend configuration files, by adding include sections

I’ve considered this as well. But does it really simplify things? Users could even create nested includes, which complicates matters further. Also, how should include order be handled if an include section appears in the middle of a config file? Do later values override those from the included file? This will become very tricky within the implementation I guess.

this creates hidden paths for configuration bits

I share the same concern. This approach offers users a lot of flexibility, at the cost of potential intransparency.
But since the concept is quite common (specially in Unix world), I think it’s a reasonable compromise and acceptable for west as well.

mbolivar · 2025-10-01T14:48:21Z

but the workspace itself is an adversarial environment in my opinion

Is the 9f4dcf5 check not enough? If not, can you elaborate on the "threat model" you have in mind?

@marc-hb it's not enough.

The threat model is basically the same as extension commands: threat actor wants to insert arbitrary code execution into something that happens when the user runs a west command in a way that is "difficult" to observe.

If you look at the way extension commands work, we only read YAML files from each west project to determine what extension command implementations it provides. We do not parse any Python when loading the manifest. If the user runs an extension command, then we parse just the python required to run it, and run it since we assume the user is opting in to running the command.

In #849 (comment), @pdgendt shows a west project with a configuration file that I understand is meant to be added to the west.configuration.Configuration object's state when an arbitrary west command, extension or otherwise, is run.

Just to illustrate where this could be problematic, consider if the adversary sets build.cmake-args to "-C pwned.cmake". Or sets alias.innocent-looking-command to forall -c "rm -rf ~". Or some other thing we add later and forget to consider in this context if you want a built-in thing and don't feel convinced since the adversary might toss other arbitrary cmake into the zephyr repo (though that is harder to hide IMO).

In summary I think it is a non-starter to allow the workspace itself to set configuration options.

marc-hb · 2025-10-01T17:02:07Z

@mbolivar I have no doubt workspace/.west/config.d/ would be a great place to do evil things. But my main question was rather: how does someone evil get there? Also, how would these new files more vulnerable than the already existing workspace/.west/config file, what's the difference? You discussed extensions and YAML files but all these generally come from the outside world through git. AFAIK that should not be the case for west config files? I mean not unless the user explicitly copies or imports some specific config files from git into his workspace, on a narrow per user, per workspace, per project and per file basis (which happens to be exactly the feature desired by @thorsten-klein[*]).

The vast majority of users have their workspace in their home directory. If something evil can write to workspace/.west/config.d/, then for the vast majority of users that means it can write anywhere in the home directory and then the user has a much bigger problem on their hand than west config files...?

[*] EDIT: users can already do this sort of "dangerous" imports with a single command like crudini --merge workspace/.west/config < evil_from_git.ini or the equivalent PowerShell command. @thorsten-klein only wants to avoid the external dependency.

mbolivar · 2025-10-01T17:14:42Z

I have no doubt workspace/.west/config.d/

I'm probably missing something -- when I see this:

$ tree -a
.
├── .west
│   ├── config
│   └── config.d
│       └── 01-generic.conf -> ../../app/shared/01-generic.conf
├── app
│   └── shared
│       └── 01-generic.conf
└── zephyr

I'm guessing you're saying that none of this is intended to create the symlink in config.d to app/shared, and the user has to do it by hand?

marc-hb · 2025-10-01T17:28:25Z

I'm guessing you're saying that none of this is intended to create the symlink in config.d to app/shared, and the user has to do it by hand?

Right, I would go even further and DISCOURAGE users to manually create such symlinks because they create a permanent bridge between a sensitive area (west configs) and updates from a less trusted one (a git repo). Instead, I would have in documentation something like: "after inspecting it very carefully, you make a (static!) copy of some .ini file into .west/config.d/"

Just for the record I think the symlink example came from @pdgendt and not for @thorsten-klein . I think it was a nice explanation shortcut to explain the rationale but with hindsight it may have backfired a little :-) EDIT: I just found @thorsten-klein did mention the symlink idea first. Does not matter.

Of course, nothing would stop "unsafe" users to create symlinks like this anyway but there's generally nothing that can be done for people driving over the speed limit without their seat belt. EDIT: and maybe it's OK if they're doing this only in a private and tightly controlled circuit. Does not matter as long as it's not officially recommended.

thorsten-klein · 2025-10-01T17:36:26Z

I’m not entirely sure where the security concerns come from at this point. The existing west config commands already allow equivalent behavior. From a user’s perspective, it makes little difference whether a script runs command west config ... or ln -s .... And if users invoke commands directly, they should understand in both cases what they are actually doing.

Would it perhaps be better to make the drop-in feature explicitly opt-in?
For example, via a setting like config.dropins = true, or even by supporting a string value such as config.dropins = ../generic-config.d:my-config.d? This would at least be safer if you are afraid that a repository ships an arbitrary .west/config.d folder (whereby I am unsure if west init will work if .west already exists).

UPDATE:
When I think about it, I like config.dropins = ../generic-config.d:my-config.d 👍🏻
It could also be supported per-type (whereby relative paths are always relative to local workspace):

[dropins]
local = ../generic-local-config.d:my-config.d
global = ../generic-global-config.d:another-global-config.d
system = ../generic-system-config.d

But this adds even more complexity. So maybe we should go with config.dropins = true.

marc-hb · 2025-10-01T18:42:26Z

From a user’s perspective, it makes little difference whether a script runs command west config ... or ln -s ...

There are (at least?) two important differences that I think concern @mbolivar (and myself now)

When you run west config..., you see what you change. You don't see the change when importing an entire file. In theory you should look at the file content too; but in practice... I'm aware this is in a nutshell the exact feature you want and I'm personally OK with it. But it makes the other difference even more critical:
A symbolic link is dynamic: the content changes every time you run west update, when you totally forgot you created this link! In other words, you surrender control of your home directory to an outside git repo. Granted, there is plenty of other attack surface in a typical build system but west is a simpler and hopefully safer tool that runs long before any build command has been issued, so the bar should be much higher. I think the bar for west alone should be as high as for git.

Thinking of it, this "dynamic" nature is also bad for reproducibility and other build properties. Imagine git bisect silently changing the build flags! Or much worse. Again: nothing can stop you from doing that but it should IMHO be discouraged.

You can admittedly symlink workspace/.west/config already but that's pretty inconvenient because it forces you to abandon the feature for yourself, so it does not come "naturally" like it would with this PR. You will almost never see that sort of advice.

So maybe we should go with config.dropins = true

Works for me.

thorsten-klein · 2025-10-03T17:02:25Z

This feature now requires explicit user activation. To enable it, users must set config.dropins = true within the config file.
Crucially, this is done on a per config file level.

For example, running west config --local config.dropins true will activate drop-in configurations only for the local config (only this west workspace).
Running west config --global config.dropins true will activate only global drop-in configurations (and no local or system ones).

This design ensures on the hand hand complete transparency: users can clearly understand that only the .d folder next to their config file is enabled.
On the other hand it also ensures security, since a globally enabled option does not automatically lead to the automatic import of local dropin configs (which are probably delivered as part of an evil west workspace).

I hope this feature is now ready 👍🏻

carlescufi · 2025-10-10T11:42:34Z

@thorsten-klein could you please provide an example of tools that already implement something similar to this? I am curious to see how others have done it and whether they've considered the security implications. Thanks!

thorsten-klein · 2025-10-10T12:29:57Z

please provide an example of tools that already implement something similar

I’ve found a few examples of tools that support drop-in configurations:

systemd: dropin.c
httpd: config.c
cron
polkit
ssh
...and many others (an AI can list quite a few more Linux tools using similar mechanisms).

However, the main difference between west and those tools is that west supports defining aliases, which can execute arbitrary commands directly (whereby other tools just allow simple config values) — for example:

[alias]
update= forall -c "rm -rf ~"

So IMO, the real security concern is not the drop-in config mechanism itself, but rather the alias feature.

Nevertheless, I believe that aliases in west are a useful and meaningful feature, so I’d like them to be supported in the drop-in configs as well.
I’m therefore happy with the compromise that drop-in configs must be explicitly enabled in the user’s configuration (at the user’s own risk).

mbolivar · 2025-10-10T16:52:45Z

So IMO, the real security concern is not the drop-in config mechanism itself, but rather the alias feature.

.... today, that we know of, in built-in west

Like I tried to explain, that was just:

an example off the top of my head
for built-in commands (west is extensible!)
for today's codebase, to which we want to add features and for which we would have to consider and safely handle the interaction with this feature forever more

thorsten-klein · 2025-10-10T17:55:09Z

@mbolivar / @carlescufi How do you want to move forward with this PR? Should I do anything else?
I’d be glad to help finalize it for merging.

marc-hb

Incomplete review. @thorsten-klein do not rush to address my comments before others agree with the general idea. I do not want you to waste time if the entire concept gets rejected.

In any case thanks for prototyping this, I think it proves this idea can fly.

marc-hb · 2025-10-14T01:31:49Z

src/west/app/config.py


+    def print_path(self, args):
+        config_path = self.config.get_path(args.configfile or LOCAL)
+        if config_path:


If the user requests the local path while outside any workspace, I would just let the tool crash. This is an obvious misuse of the tool, no need to test for that in the code.

I found a raise MalformedConfig('local configuration file not found') somewhere else, that's another option.

In any case I would never exit 0, the tool should not succeed.

PS: please do not reply with a question and then hide the thread as "resolved" :-D

marc-hb · 2025-10-14T01:36:06Z

src/west/configuration.py


+    def get_path(self, configfile: ConfigFile = ConfigFile.LOCAL):
+        if configfile == ConfigFile.ALL:
+            raise RuntimeError(f'{configfile} not allowed for get_path')


Suggested change

raise RuntimeError(f'{configfile} not allowed for get_path')

raise RuntimeError(f'{configfile} not allowed for --print-path')

marc-hb · 2025-10-14T01:44:08Z

src/west/configuration.py

+        cf = _InternalCF(path) if path else None
+        if not cf:
+            return None
+        if not cf.path and not cf.dropin_paths:


Why bother creating an object when neither exists?

Good point. It can be changed to

if not path: return None cf = _InternalCF(path)

marc-hb · 2025-10-14T01:46:39Z

src/west/configuration.py

+        if self.path:
+            self.cp.read(self.path, encoding='utf-8')
+        if self.dropin_paths:
+            self.dropin_cp.read(self.dropin_paths, encoding='utf-8')


Please explain in the class _InternalCF: "pydoc" why you need to keep these in separate configparser objects. Because appending is tricky maybe?

At first sight, it looks like merging both here (with the appropriate precedence) would save a ton of code below.

Exerting precedence here would be a good place to let the user know with some INFO log statement.

Either way, the class _InternalCF becomes a bit more complicated and needs some introduction explaining its data structures.

marc-hb · 2025-10-14T01:47:37Z

src/west/configuration.py

+
+    def _write(self):
+        if not self.path:
+            raise WestNotFound('No config file exists that can be written')


If not too difficult, it would be nice to print the missing path in the error message.
Also, if dropins are enabled then it would be nice to print a reminder that writing and appending ignores them.

I think this can be refactored/simplified again since with current concept a config file must always exist (where the dropin config is actually enabled).

You are right, that a reminder would be nice. You prefer it only in case of writing to a config via command line, or always on reading (e.g. on stderr)?

marc-hb · 2025-10-14T01:54:46Z

src/west/configuration.py

        section, key = _InternalCF.parse_key(option)
-
-        if section not in self.cp:
+        if (section not in self.cp) or (key not in self.cp[section]):


Again, maybe a reminder that dropins are ignored.

marc-hb · 2025-10-14T01:58:04Z

src/west/configuration.py

+                    if section not in cp:
+                        cp.add_section(section)
+                    for key, value in contents.items():
+                        cp[section][key] = value


Would you mind expanding a bit the pydoc for this function? It's pretty cryptic to me, sorry. Or just add some relevant references.

Honestly I am not sure if this function is still neccessary. I have removed it completely (and its usage in main.py), and the tests still pass.

The comment at the start of the function would indicate that west extension commands could rely on it?

# Internal API for main to use to maintain backwards # compatibility for existing extensions using the legacy # function-and-global-state APIs.

Okay. Is it possible that there is no test existing for this? I am not sure how I can guarantee that I do not break something here, since I do not know the previous behavior.
So does it make sense that I adapt this function? Will such a function be kept forever or will it be removed at some time?

The answers to this sort of questions usually lie in git history. git blame works but I tend to prefer tig blame. GitHub's blame interface is surprisingly decent and it has direct links to the corresponding pull requests. Of course, this assumes the git history is usable with no too much whitespace pollution :-)

Maybe this? Just spotted by chance:
https://docs.zephyrproject.org/4.2.0/develop/west/west-apis.html#configuration-api

... recommended API to use since west v0.13.

marc-hb · 2025-10-14T02:02:48Z

src/west/app/config.py

+and system), whereby all '.conf' files from each dropin directory are loaded in
+alphabetical order.
+For example:
+    .west/config.d/basics.conf


I think this falls a bit short and misses at least the fact that writing and appending ignores dropins.

pdgendt · 2025-10-14T08:00:50Z

I'm going to be honest here, that I'm not keen on adding these configuration dropins. The configuration mechanism for west developers (and a bit for west users) is a lot more complicated. Future changes need be aware to not break any of this.

Can you provide some insight into where you "really" need this kind of flexibility?

In our company, only a small group of people maintains the build system and tooling and configs (e.g., west). Their goal is to set up the development environment correctly and shield everyone else from pitfalls. Similarly to this, also customer repositories may define configurations (on top of the generic configs) that should be shared across their product developers to ensure consistency. This leads to the need for maintaining (generic) configs and a mechanism to overlay configs. In the end, maybe something like a "repository config level" is missing for us.

Mind you there's already a mechanism for extending functionality: extension commands. You can simply provide west extension commands that are "simple" wrapper commands.
Correct me if I'm wrong, but I think you've pointed out sharing alias commands, which can be done just as easy with extension commands.

thorsten-klein · 2025-10-14T14:42:46Z

I'm going to be honest here, that I'm not keen on adding these configuration drop-ins.

Thanks for the honest feedback! 👍🏻

Correct me if I'm wrong, but I think you've pointed out sharing alias commands:

I assume we’re not the first ones who’d like to share a company-wide west configuration within a repository, are we?

That was just one example that came to mind. For instance, we’d also want to ensure that auto-cache is enabled, or define a default build.board.

On the other hand, I also agree that this relatively small use case might not justify adding a large feature like configuration drop-ins. So I am also open for any other solutions for our use case.

So, to summarize our use case again:

At our company, we’d like to maintain a company-wide west configuration with default values.
We want to make sure some config values are always set — either explicitly by the user in their local config, or by falling back to the defaults from our company-wide config.
Of course, users must always be able to override any company-wide values.

Without a “drop-in configs” feature, here’s how we tried to approach it:

Approach 1:
We could check in our .west/config file into the repository to share it across the team.
The downside is that any local change by a user would appear as a Git diff.
Additionally, nested .west folders don’t work well (for example, if this repository is later integrated into another app).
Result: This is not a good idea.

Approach 2:
We have a bootstrapping script that runs west init, west update. This script could also call west config section.key value to ensure the expected values are set. However, this command overwrites existing values — no matter if the user has intentionally changed them, or if they simply have outdated values (previous defaults).
What would be your recommendation for handling this?
Result: This is not a good idea.

Do you see the challenge we’re facing? What would your suggested (existing) solution?

marc-hb · 2025-10-14T15:18:55Z

I’ve found a few examples of tools that support drop-in configurations:

As mentioned before in #847, sudo find /etc /usr -name '*.d' -type d shows more than 100 examples on one of my linux systems. This "divide and conquer" configuration approach has quickly become universal; @thorsten-klein's issue is really not special. Most Linux kernel builds are configured with Kconfig "fragments"; the kernel has scripts/kconfig/merge_config.sh and other scripts for that. From https://docs.zephyrproject.org/latest/build/snippets/design.html

able to combine multiple types of configuration: snippets make it possible to store multiple different types of build system settings in one place, and apply them all together

Dropins are part of the bigger "configuration as code" approach; they enable it.

While not exactly it, the flexibility to move files in and out of .d directories would be a useful alternative to solve the #429 use case.

Added new flag `config --print-path` (`-p`) for easy determination of the currently active configuration. The path is printed on console.

pdgendt · 2025-10-14T20:12:41Z

src/west/configuration.py

+                    if conf.suffix in ['.conf', '.ini']:
+                        self.dropin_paths.append(self.dropin_dir / conf)


Why also have .ini files included? This is not how it's documented above

All files inside a drop-in directory must use `.conf` extension and are loaded in **alphabetical order**.

pdgendt · 2025-10-14T20:20:10Z

src/west/configuration.py

+        if section in self.cp and key in self.cp[section]:
+            getter = config_getter
+        elif section in self.dropin_cp and key in self.dropin_cp[section]:
+            getter = dropin_getter
+        else:
+            raise KeyError(option)


There is a subtle difference here, but one nonetheless. Before we were able to determine a difference between a missing key or an entire missing section, we no longer have that difference.

pdgendt · 2025-10-14T20:25:00Z

src/west/configuration.py

+                    if section not in cp:
+                        cp.add_section(section)
+                    for key, value in contents.items():
+                        cp[section][key] = value


The comment at the start of the function would indicate that west extension commands could rely on it?

# Internal API for main to use to maintain backwards # compatibility for existing extensions using the legacy # function-and-global-state APIs.

Added new flag `config --print-path` (`-p`) for easy determination of the currently active configuration. The path is printed on console. '--print-path' may display default configuration paths for system and global configurations. The local configuration must either exist or be specified via environment variable, since it cannot be determined otherwise.

added config command to list all config files and dropin files which are currently considered by west.

config files in acccording dropin directories are considered automatically. The dropin directory is named as the config itself but with a '.d' suffix. For example for local config the dropin directory is './west/config.d' and for global config it is '~/.westconfig.d' support config dropin directories config files in acccording dropin directories are considered automatically. The dropin directory is named as the config itself but with a '.d' suffix. For example for local config the dropin directory is './west/config.d' and for global config it is '~/.westconfig.d' Boolean option `config.dropins` must be set to `true` so that dropin configs for the according config type are considered. So if the option config.dropins=true in enabled in local config, only local dropin configs are used (but no system or global ones).

thorsten-klein force-pushed the west-config-d branch 3 times, most recently from 682c916 to e80b867 Compare September 27, 2025 06:44

pdgendt self-requested a review September 27, 2025 10:25

pdgendt reviewed Sep 27, 2025

View reviewed changes

src/west/configuration.py Outdated Show resolved Hide resolved

thorsten-klein force-pushed the west-config-d branch 3 times, most recently from df23c63 to 883bc9b Compare September 27, 2025 14:31

pdgendt reviewed Sep 29, 2025

View reviewed changes

pdgendt requested a review from carlescufi September 29, 2025 11:52

marc-hb reviewed Sep 30, 2025

View reviewed changes

pdgendt requested a review from mbolivar September 30, 2025 06:48

thorsten-klein force-pushed the west-config-d branch from 883bc9b to 42bc614 Compare October 1, 2025 07:53

thorsten-klein force-pushed the west-config-d branch from 42bc614 to 3572a90 Compare October 1, 2025 10:45

thorsten-klein force-pushed the west-config-d branch from c29931f to de473e3 Compare October 1, 2025 17:53

thorsten-klein force-pushed the west-config-d branch from bab29c2 to 8577c96 Compare October 3, 2025 16:23

thorsten-klein requested a review from marc-hb October 3, 2025 16:29

thorsten-klein requested a review from pdgendt October 3, 2025 16:29

thorsten-klein force-pushed the west-config-d branch from 8577c96 to 4b0e0b6 Compare October 3, 2025 16:31

thorsten-klein force-pushed the west-config-d branch 5 times, most recently from 741eff6 to 32103d4 Compare October 9, 2025 15:54

thorsten-klein mentioned this pull request Oct 10, 2025

Support 4th level west -c config.ini file(s) #429

Open

thorsten-klein changed the title ~~Automatically use additional config files from .d directories~~ Support dropin config files (within .d directories) Oct 10, 2025

marc-hb mentioned this pull request Oct 14, 2025

configuration.py: add types to _whence() function, other cosmetic fixes #866

Open

marc-hb reviewed Oct 14, 2025

View reviewed changes

support 'config --print-path'

c032bc3

Added new flag `config --print-path` (`-p`) for easy determination of the currently active configuration. The path is printed on console.

thorsten-klein force-pushed the west-config-d branch from 32103d4 to f475a52 Compare October 14, 2025 20:31

pdgendt reviewed Oct 14, 2025

View reviewed changes

thorsten-klein force-pushed the west-config-d branch from f475a52 to 9ad9bdb Compare October 15, 2025 13:20

thorsten-klein mentioned this pull request Oct 15, 2025

[alternative] Support multiple west config files per config level #867

Draft

thorsten-klein force-pushed the west-config-d branch from 9ad9bdb to f624d2f Compare October 15, 2025 14:23

thorsten-klein added 3 commits October 16, 2025 08:28

add support for 'config --list-paths'

eac7ccc

added config command to list all config files and dropin files which are currently considered by west.

thorsten-klein force-pushed the west-config-d branch from f624d2f to b9ee7fc Compare October 16, 2025 06:35

	raise RuntimeError(f'{configfile} not allowed for get_path')
	raise RuntimeError(f'{configfile} not allowed for --print-path')

		if conf.suffix in ['.conf', '.ini']:
		self.dropin_paths.append(self.dropin_dir / conf)

Support dropin config files (within .d directories) #849

Are you sure you want to change the base?

Support dropin config files (within .d directories) #849

Uh oh!

Conversation

thorsten-klein commented Sep 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why?

Solution

Uh oh!

codecov bot commented Sep 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pdgendt left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pdgendt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thorsten-klein Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thorsten-klein Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thorsten-klein commented Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thorsten-klein Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pdgendt commented Sep 30, 2025

Uh oh!

mbolivar commented Sep 30, 2025

Uh oh!

marc-hb commented Oct 1, 2025

Uh oh!

pdgendt commented Oct 1, 2025

Uh oh!

thorsten-klein commented Oct 1, 2025

Uh oh!

mbolivar commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

marc-hb commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mbolivar commented Oct 1, 2025

Uh oh!

marc-hb commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Support dropin config files (within `.d` directories) #849

Support dropin config files (within `.d` directories) #849

thorsten-klein commented Sep 27, 2025 •

edited

Loading

codecov bot commented Sep 27, 2025 •

edited

Loading

thorsten-klein Sep 29, 2025 •

edited

Loading

thorsten-klein Sep 29, 2025 •

edited

Loading

thorsten-klein commented Sep 29, 2025 •

edited

Loading

thorsten-klein Oct 1, 2025 •

edited

Loading

mbolivar commented Oct 1, 2025 •

edited

Loading

marc-hb commented Oct 1, 2025 •

edited

Loading

marc-hb commented Oct 1, 2025 •

edited

Loading

thorsten-klein commented Oct 1, 2025 •

edited

Loading

marc-hb commented Oct 1, 2025 •

edited

Loading