Skip to content

Move ownership of Variables from Application to Profile#759

Open
adriaan42 wants to merge 1 commit into
redhat-performance:masterfrom
adriaan42:adriaan/variable-bug
Open

Move ownership of Variables from Application to Profile#759
adriaan42 wants to merge 1 commit into
redhat-performance:masterfrom
adriaan42:adriaan/variable-bug

Conversation

@adriaan42

@adriaan42 adriaan42 commented Mar 10, 2025

Copy link
Copy Markdown
Contributor

While working on #758, I noticed a bug in the way variables are handled:

Currently, there is one global Variables object, as member of the global Application. When loading profiles, variables are added to it, but never removed. So when switching profiles, there are still variables present from the previous profile.

To demonstrate, use this debug print, with two profiles:

diff --git a/tuned/daemon/daemon.py b/tuned/daemon/daemon.py
index a902dec..fcba9cd 100644
--- a/tuned/daemon/daemon.py
+++ b/tuned/daemon/daemon.py
@@ -335,6 +335,8 @@ class Daemon(object):
                if self._profile is None:
                        return False
 
+               log.debug(f"BUG: variables={self._profile_loader._variables._lookup_re.keys()}")
+
                log.info("starting tuning")
                self._not_used.set()
                self._thread = threading.Thread(target=self._thread_code)
  • Profile variable1:
    [variables]
    foo=42
  • Profile variable2:
    [variables]
    bar=23

Then use tuned-adm profile to switch between them, and note how the debug output will contain not only variables of the current, but also of the previously loaded profile:

[... started with profile variable1 active ...]
2025-03-10 08:58:57,683 INFO     tuned.profiles.loader: loading profile: variable1
2025-03-10 08:58:57,688 DEBUG    tuned.daemon.daemon: BUG: variables=dict_keys(['(?<!\\\\)\\${foo}'])
2025-03-10 08:58:57,688 INFO     tuned.daemon.daemon: starting tuning
[... tuned-adm profile variable2 ...]
2025-03-10 08:59:21,570 INFO     tuned.profiles.loader: loading profile: variable2
2025-03-10 08:59:21,571 DEBUG    tuned.daemon.daemon: BUG: variables=dict_keys(['(?<!\\\\)\\${foo}', '(?<!\\\\)\\${bar}'])
2025-03-10 08:59:21,571 INFO     tuned.daemon.daemon: starting tuning

My proposal is to have the Variables object owned by the Profile. It is created by the profile loader, so that more complex load scenarios (multiple profiles, potentially with includes) still only use one Variables object.

An alternative would be to simply clear variables when loading a new profile, but that would break my #758, where (when restoring a snapshot) a second profile is loaded, which can fail, and therefore does not work well with Variables as global state.

@adriaan42 adriaan42 force-pushed the adriaan/variable-bug branch from a531df0 to f17c247 Compare June 8, 2026 10:19
@coderabbitai

coderabbitai Bot commented Jun 8, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Summary by CodeRabbit

  • Refactor
    • Improved internal variable handling across the plugin instance configuration system to enhance variable context management throughout the application.

Walkthrough

This PR refactors variable handling throughout tuned by moving variables from plugin-level storage to per-instance variables. Profile/Factory/Loader/Merger are updated to thread and merge Variables per-load, Instance stores per-instance Variables, Plugin/Repository/Application/Manager/Daemon signatures and wiring were changed to pass variables, sysfs and command expansions use instance variables, and tests were updated to match new signatures.

Changes

Per-Instance Variable Refactoring

Layer / File(s) Summary
Profile data model and variable configuration
tuned/profiles/profile.py
Profile constructor now requires a variables argument, stores _variables and _variable_cfg (OrderedDict), and exposes variable_cfg alongside variables.
Profile factory and loading with variables threading
tuned/profiles/factory.py, tuned/profiles/loader.py, tuned/profiles/merger.py
Factory.create now accepts variables and passes it to Profile. Loader no longer stores loader-level variables, instantiates a per-load Variables() and threads it through profile creation/loading/includes, then merges variable_cfg into final profile. Merger reuses/creates a shared Variables and merges variable_cfg with prepend/overwrite behavior.
Merger variable_cfg handling
tuned/profiles/merger.py
merge() seeds reduce with Profile(..., variables); _merge_two updates/clears profile_a.variable_cfg, merges overlapping keys, and reorders on prepend.
Instance class variables storage
tuned/plugins/instance/instance.py
Instance.init accepts variables, assigns to self._variables, and exposes a read-only variables property.
Plugin base class per-instance variable usage
tuned/plugins/base.py
Plugin.init drops variables. create_instance accepts variables and forwards to instance factory. Variable expansion and script env composition now use instance._variables (expand/get_env) instead of plugin-level variables.
Repository and Application wiring updates
tuned/plugins/repository.py, tuned/daemon/application.py
Repository no longer accepts/stores variables and passes global_cfg when instantiating plugins. Application no longer initializes or passes self.variables to Repository or Loader.
Daemon and Manager coordinate variable passing
tuned/daemon/daemon.py, tuned/units/manager.py
Daemon._thread_code passes the selected profile's variables to unit_manager.create; Manager.create signature extended to accept variables and forwards it to each plugin.create_instance call.
SysfsPlugin uses per-instance variables
tuned/plugins/plugin_sysfs.py
_instance_apply_static and _instance_verify_static expand values using instance.variables.expand(...) instead of plugin-level variables.
Test updates across all components
tests/unit/plugins/test_base.py, tests/unit/profiles/test_loader.py, tests/unit/profiles/test_merger.py, tests/unit/profiles/test_profile.py
Tests updated to match new signatures: Loader/CommandsPlugin constructor calls adjusted, create_instance(...) calls now include the variables argument (None or Variables()), and Profile/MockProfile constructions in tests receive a variables argument where required.

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 16.07% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main architectural change: moving the ownership of Variables from the Application class to the Profile class.
Description check ✅ Passed The description provides a detailed explanation of the bug (variables not being removed when switching profiles), demonstrates it with concrete examples, and justifies the proposed solution.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
tuned/profiles/loader.py (1)

26-27: 💤 Low value

_create_profile appears unused.

This method creates a Profile directly, but _load_profile at line 80 uses self._profile_factory.create() instead. If this is intentional (e.g., for subclass overriding), consider documenting the design intent. Otherwise, it may be dead code.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tuned/profiles/loader.py` around lines 26 - 27, _dead code: _create_profile
is defined but never used; either remove it or integrate/document its intended
override use. If intended for subclassing, add documentation on the class or the
_create_profile method stating that subclasses should override _create_profile
to customize Profile construction and change _load_profile to call
self._create_profile(...) instead of self._profile_factory.create(...). If not
intended, delete the _create_profile method and rely solely on the
_profile_factory.create(...) usage found in _load_profile. Ensure references to
tuned.profiles.profile.Profile are adjusted or removed accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tuned/plugins/base.py`:
- Around line 251-253: The code currently aliases and mutates os.environ via the
local variable environ (environ = os.environ;
environ.update(instance._variables.get_env())), which pollutes the global
process environment; instead create a fresh per-call environment dict (e.g., env
= os.environ.copy()), update that with instance._variables.get_env(), and use
that env only for the subprocess/script invocation (where arguments is used) so
you do not modify os.environ globally. Ensure you replace uses of the aliased
environ with the new env when launching the process.

---

Nitpick comments:
In `@tuned/profiles/loader.py`:
- Around line 26-27: _dead code: _create_profile is defined but never used;
either remove it or integrate/document its intended override use. If intended
for subclassing, add documentation on the class or the _create_profile method
stating that subclasses should override _create_profile to customize Profile
construction and change _load_profile to call self._create_profile(...) instead
of self._profile_factory.create(...). If not intended, delete the
_create_profile method and rely solely on the _profile_factory.create(...) usage
found in _load_profile. Ensure references to tuned.profiles.profile.Profile are
adjusted or removed accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 1d8d090d-374f-4e7b-bbdd-6fc0a800ce41

📥 Commits

Reviewing files that changed from the base of the PR and between 0eb28ac and f17c247.

📒 Files selected for processing (15)
  • tests/unit/plugins/test_base.py
  • tests/unit/profiles/test_loader.py
  • tests/unit/profiles/test_merger.py
  • tests/unit/profiles/test_profile.py
  • tuned/daemon/application.py
  • tuned/daemon/daemon.py
  • tuned/plugins/base.py
  • tuned/plugins/instance/instance.py
  • tuned/plugins/plugin_sysfs.py
  • tuned/plugins/repository.py
  • tuned/profiles/factory.py
  • tuned/profiles/loader.py
  • tuned/profiles/merger.py
  • tuned/profiles/profile.py
  • tuned/units/manager.py

Comment thread tuned/plugins/base.py Outdated
@adriaan42 adriaan42 changed the title Draft: Bug in Variable handling Move ownership of Variables from Application to Profile Jun 8, 2026
Currently, there is one global Variables object, as member of the global
Application. When loading profiles, variables are added to it, but never
removed. So when switching profiles, there are still variables present
from the previous profile.
With this commit, the Variables object is now owned by the Profile.
It is created by the profile loader, so that more complex laod scenarios
(multiple profiles, potentially with includes) still only use one Variables
object.

Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
@adriaan42 adriaan42 force-pushed the adriaan/variable-bug branch from f17c247 to b863272 Compare June 8, 2026 11:17

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
tests/unit/plugins/test_base.py (1)

106-116: ⚡ Quick win

Exercise actual instance-variable expansion in these tests.

These cases wire the new parameter through, but they still use literal command values, so they would pass even if runtime expansion never read instance.variables. Seed a Variables object and use ${...} in the options here, and in test_get_current_value, so this PR gets a regression test for the behavior it changes.

This aligns with the PR objective to move runtime expansion to per-instance variables.

Also applies to: 135-141

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unit/plugins/test_base.py` around lines 106 - 116, The tests currently
pass literal option values so they don't validate per-instance variable
expansion; update the test cases (those calling create_instance and invoking
_execute_all_non_device_commands and test_get_current_value) to seed a
profiles.variables.Variables() instance with the appropriate key/value and
change the option maps to use a ${variable_name} reference (e.g.,
{'size':'${size}'} and {'device_setting':'${device_setting}'}), then assert that
after calling _execute_all_non_device_commands (and the device command test) the
plugin resolved those variables into the expected concrete values (e.g., _size
== 'XXL' and device setting == '010'); reference create_instance and
_execute_all_non_device_commands to locate where to pass the Variables instance
and where expansion should occur, and apply the same pattern to the similar
tests around the 135-141 region.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@tests/unit/plugins/test_base.py`:
- Around line 106-116: The tests currently pass literal option values so they
don't validate per-instance variable expansion; update the test cases (those
calling create_instance and invoking _execute_all_non_device_commands and
test_get_current_value) to seed a profiles.variables.Variables() instance with
the appropriate key/value and change the option maps to use a ${variable_name}
reference (e.g., {'size':'${size}'} and {'device_setting':'${device_setting}'}),
then assert that after calling _execute_all_non_device_commands (and the device
command test) the plugin resolved those variables into the expected concrete
values (e.g., _size == 'XXL' and device setting == '010'); reference
create_instance and _execute_all_non_device_commands to locate where to pass the
Variables instance and where expansion should occur, and apply the same pattern
to the similar tests around the 135-141 region.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 30d9634b-d09c-403f-86e1-5c3e26c0301c

📥 Commits

Reviewing files that changed from the base of the PR and between f17c247 and b863272.

📒 Files selected for processing (15)
  • tests/unit/plugins/test_base.py
  • tests/unit/profiles/test_loader.py
  • tests/unit/profiles/test_merger.py
  • tests/unit/profiles/test_profile.py
  • tuned/daemon/application.py
  • tuned/daemon/daemon.py
  • tuned/plugins/base.py
  • tuned/plugins/instance/instance.py
  • tuned/plugins/plugin_sysfs.py
  • tuned/plugins/repository.py
  • tuned/profiles/factory.py
  • tuned/profiles/loader.py
  • tuned/profiles/merger.py
  • tuned/profiles/profile.py
  • tuned/units/manager.py
✅ Files skipped from review due to trivial changes (1)
  • tests/unit/profiles/test_profile.py
🚧 Files skipped from review as they are similar to previous changes (11)
  • tuned/plugins/plugin_sysfs.py
  • tuned/plugins/instance/instance.py
  • tuned/daemon/daemon.py
  • tuned/units/manager.py
  • tuned/daemon/application.py
  • tuned/plugins/base.py
  • tests/unit/profiles/test_loader.py
  • tuned/profiles/factory.py
  • tests/unit/profiles/test_merger.py
  • tuned/profiles/merger.py
  • tuned/plugins/repository.py

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant