Skip to content

Conversation

thorsten-klein
Copy link
Contributor

Fixes #862

A new pytest fixture is added which is automatically applied (autouse=True).
This fixture sets the west environment variables WEST_CONFIG_SYSTEM and WEST_CONFIG_GLOBAL during each test, in order to prevent it from using or modifying the user's actual global or system configuration.

Note: The environment variables are inherited from subprocess, so that also the subprocess calls do not use or touch the user's real configs.

@thorsten-klein
Copy link
Contributor Author

@pdgendt We could also create some invalid user config in the CI system (at the default path) to ensure that this corrupt config is never used in CI. Otherwise tests could potentially touch real configs again in future (e.g. when they use del os.environ['WEST_CONFIG_GLOBAL'])

Copy link

codecov bot commented Oct 9, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.34%. Comparing base (a3af732) to head (aa0b156).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #864      +/-   ##
==========================================
- Coverage   84.52%   84.34%   -0.18%     
==========================================
  Files          11       11              
  Lines        3366     3366              
==========================================
- Hits         2845     2839       -6     
- Misses        521      527       +6     

see 3 files with indirect coverage changes

Copy link
Collaborator

@pdgendt pdgendt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fast fix! This looks good IMO.

@pdgendt
Copy link
Collaborator

pdgendt commented Oct 9, 2025

@pdgendt We could also create some invalid user config in the CI system (at the default path) to ensure that this corrupt config is never used in CI. Otherwise tests could potentially touch real configs again in future (e.g. when they use del os.environ['WEST_CONFIG_GLOBAL'])

I like the idea, would even be better if we could verify that locally before CI, but not sure if that's easy.

@thorsten-klein thorsten-klein force-pushed the isolate-tests-from-user-config branch from 13052d0 to 0ed2765 Compare October 9, 2025 11:22
Copy link
Collaborator

@pdgendt pdgendt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some styling fixes

@thorsten-klein
Copy link
Contributor Author

I have added this new method update_env() which can be safely used in tests in order to unset env variables (instead of del os.environ). Maybe there is some way to forbid via conftest.oy that del os.environ is used?

@thorsten-klein thorsten-klein force-pushed the isolate-tests-from-user-config branch from 0ed2765 to 0dfd479 Compare October 9, 2025 11:38
@pdgendt pdgendt requested a review from marc-hb October 9, 2025 11:45
@thorsten-klein
Copy link
Contributor Author

I will rebase once your PR #865 is merged and run poe format 👍

@pdgendt
Copy link
Collaborator

pdgendt commented Oct 9, 2025

I will rebase once your PR #865 is merged and run poe format 👍

Update this PR instead, so we can go ahead with this one. I'm afraid the formatting PR will need some more eyes and time.

@thorsten-klein thorsten-klein force-pushed the isolate-tests-from-user-config branch from 0dfd479 to 476f4ae Compare October 9, 2025 14:08
@pdgendt pdgendt requested review from carlescufi and mbolivar October 9, 2025 14:20
@marc-hb
Copy link
Collaborator

marc-hb commented Oct 9, 2025

I haven't looked at the code yet but it reminded me 6cf0a02... was this lost in the transition to uv?

@pdgendt
Copy link
Collaborator

pdgendt commented Oct 9, 2025

I haven't looked at the code yet but it reminded me 6cf0a02... was this lost in the transition to uv?

I don't think so, it just wasn't noticed when new tests were added.

@pdgendt
Copy link
Collaborator

pdgendt commented Oct 10, 2025

I haven't looked at the code yet but it reminded me 6cf0a02... was this lost in the transition to uv?

I don't think so, it just wasn't noticed when new tests were added.

Can confirm this issue exists in v1.5.0 running with tox.

@thorsten-klein
Copy link
Contributor Author

We could also create some invalid user config in the CI system (at the default path) to ensure that this corrupt config is never used in CI

Let's tackle this in a follow-up PR 👍

Once a second reviewer has approved, this PR can be merged, correct?

@thorsten-klein thorsten-klein force-pushed the isolate-tests-from-user-config branch from 476f4ae to c565af6 Compare October 10, 2025 13:18
@pdgendt pdgendt requested a review from Copilot October 10, 2025 16:03
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces test isolation for west configuration files to prevent tests from interfering with user's actual global and system configuration. The main change adds an autouse pytest fixture that sets environment variables to redirect config file locations during test execution.

Key changes:

  • Added autouse fixture to isolate global and system config files during tests
  • Refactored existing config test fixtures to use new context managers
  • Introduced reusable update_env and chdir context managers

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
tests/conftest.py Added autouse fixture for config isolation and utility context managers
tests/test_config.py Updated to use new update_env context manager
tests/test_manifest.py Refactored to use update_env instead of manual environment handling

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@thorsten-klein thorsten-klein force-pushed the isolate-tests-from-user-config branch from c565af6 to 99c1f84 Compare October 10, 2025 16:29
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@thorsten-klein thorsten-klein force-pushed the isolate-tests-from-user-config branch from 99c1f84 to 2f34090 Compare October 10, 2025 16:41
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.


Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@pdgendt
Copy link
Collaborator

pdgendt commented Oct 13, 2025

@mbolivar PTAL, as you're the original author of the environment mechanisms in tests.

I think the fix is elegant and more Pythonic.

Copy link
Collaborator

@marc-hb marc-hb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a quick look and spotted nothing wrong, thanks for the fix.

thorsten-klein and others added 2 commits October 16, 2025 07:33
Add a new automatically applied pytest fixture to prevent tests from
using or modifying the user's actual global or system configuration.
To support this, two context manager helpers are added:
- `update_env(env)` — temporarily updates environment variables within a
`with` block
- `chdir(path)` — temporarily changes the current working directory
within a `with` block
@thorsten-klein thorsten-klein force-pushed the isolate-tests-from-user-config branch from 3bdab91 to aa0b156 Compare October 16, 2025 05:34
@pdgendt pdgendt merged commit a53dbf4 into zephyrproject-rtos:main Oct 16, 2025
26 checks passed
@thorsten-klein thorsten-klein deleted the isolate-tests-from-user-config branch October 16, 2025 07:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Running test suite accesses user and system configuration files

4 participants