tests/toolbox: adjust to be runnable on RHCOS by Rolv-Apneseth · Pull Request #4017 · coreos/fedora-coreos-config

Rolv-Apneseth · 2026-02-18T13:47:28Z

This will should enable running the basic toolbox test for RHCOS.

To explain a couple of the reasons it would not work in its current state in RHCOS, and the changes this PR makes to get around them:

No machinectl command on RHCOS. I've instead used su - core to run commands as the core user. Also has the minor benefit of propagating error codes, though I opted to keep the explicit checks anyway.
Grep and awk calls were looking only for fedora-toolbox-. I simply expanded the search to also accept rhel-toolbox-.
The rhel-9.8 variant is using this toolbox - this is the one that adds the most complexity to this script. Commands for this one are completely different, the script uses root podman and requires extra configuration. I have a butane config to setup the tool as required, and I've defined and overridden functions to test the same functionality across this script and the modern toolbox binary.

I would love feedback, as I'm not sure all the decisions I made were the best ones. For example, I could see a case for keeping the machinectl path for systems which have it available, as it actually simulates an interactive login. Also, do we even need to test that older toolbox script? Either way, I'm happy to make any changes.

gemini-code-assist

Code Review

This pull request successfully adapts the toolbox test to be runnable on RHCOS by replacing machinectl with su - core, expanding search patterns for toolboxes, and cleverly handling an older toolbox script via function overrides. The changes are well-structured and make the test more robust. I've identified a bug in the toolbox_count function that could cause the script to fail, and I've also suggested a refactoring to improve the clarity of the code that handles the older toolbox version. Overall, this is a great improvement.

tests/kola/toolbox/test.sh

Rolv-Apneseth · 2026-02-18T19:33:15Z

Anybody know why that CI is failing?

./tmt-test-wrapper-inner.sh-script-00-1-default-0-default-0: line 1: tmt: command not found

Also, should these be pointing at a fork? I assume this was for testing / debugging:

git clone --recurse-submodules https://github.com/Roshan-R/fedora-coreos-config testcode
...
git -C testcode fetch https://github.com/Roshan-R/fedora-coreos-config kola-testing-farm:gluetool/kola-testing-farm

Rolv-Apneseth · 2026-03-20T09:17:46Z

/retest

dustymabe · 2026-03-18T13:01:39Z

tests/kola/toolbox/config.bu

+version: 1.6.0
+storage:
+  files:
+    # Toolbox configuration for older streams using https://github.com/coreos/toolbox


What do you mean by this exactly?

Yeah sorry, wasn't sure how to word it. I suppose it's only RHCOS 9.x and older that use the coreos toolbox

How about

Toolbox configuration for RHCOS variants < 10.x using https://github.com/coreos/toolbox

ok I see. So this is wholly for the legacy toolbox?

TBH in that case I would just put it in the test itself for the legacy case:

cat <<EOF > /var/home/core/.toolboxrc REGISTRY=quay.io IMAGE=fedora/fedora-toolbox:latest TOOLBOX_NAME=fedora-toolbox-latest AUTHFILE=/dev/null EOF

Unfortunately, RHCOS 10 also use the legacy coreos/toolbox implementation. So it's a RHCOS vs FCOS thing right now.

When I was testing it was only 9.8 that had it, not even c9s.

Here's the toolbox package metadata from the release browser for each (not sure where else to get a list of packages):

9.8: [ "toolbox", "0", "0.1.2", "1.rhaos4.22.el9", "noarch" ]

10.0: [ "toolbox", "0", "0.3", "1.el10", "x86_64" ]

c9s: [ "toolbox", "0", "0.3", "1.el9", "x86_64" ]

c10s: [ "toolbox", "0", "0.3", "1.el10", "x86_64" ]

dustymabe · 2026-03-18T13:02:01Z

tests/kola/toolbox/config.bu

+    - path: /var/home/core/config.json
+      contents:
+        inline: "{}"


Not sure I understand the point of this

I will double check but I believe when I was running it, the coreos toolbox would fail if the file didn't exist

Yeah, it fails without that. Either:

Error: credential file is not accessible: faccessat /var/lib/kubelet/config.json: no such file or directory

Or, if specifying AUTHFILE:

Error: credential file is not accessible: faccessat /var/home/core/config.json: no such file or directory

Added a comment

ok so you mean coreos/toolbox fails if a credentials file doesn't exist? Does it work if you use AUTHFILE=/dev/null ?

Unfortunately not:

Error: unable to copy from source docker://quay.io/fedora/fedora-toolbox:latest: initializing source docker://quay.io/fedora/fedora-toolbox:latest: getting username and password: reading JSON file "/dev/null": unmarshaling JSON at "/dev/null": unexpected end of JSON input

And yes, this whole butane file is only for coreos/toolbox

dustymabe · 2026-03-20T12:39:16Z

tests/kola/toolbox/test.sh


+# IMPORTANT: Commands are run indirectly via `su - core` to re-create the
+# user environment needed for unprivileged podman functionality.
+run_as_core()


we use a similar function to this in a few other tests (search for runascoreuser). might be a good idea to put a function like this in commonlib.sh and then use that everywhere.

migt be good to point AI at this and it could do it in like 10s.

Note though.. that machinectl shell is a more proper way to get an environment that more closely models what a user session would have. Maybe we should continue to use machinectl shell here.

cc @travier

oh - hmm. I now see the comment above that mentions RHCOS doesn't have machinectl. I wonder if we should just add it.

Yes that was a big part of the complication with getting it to run on rhcos. I did consider adding it to the common lib but I wanted to see if we were gonna keep the machinectl approach for systems that support it first

yeah, let's see what @travier thinks on the machinectl in RHCOS thing.

I would prefer we keep using machinectl on FCOS. I think I had tried with su, I don't remember why it did not work.

I think my question was more: should we add machinectl to RHCOS?

I don't think so if we don't need it

if we don't need it for something else. I don't remember why it's in FCOS

I had confirmed this works on both, which is why I didn't choose to split it here, but I don't mind keeping machinectl for FCOS if we think it's worth doing it the more "proper" way when we can

tests/kola/toolbox/test.sh

Fixes openshift/os#1035 RHCOS incompatabilities and adjustments made by this commit: 1. No `machinectl` command on RHCOS, so instead use `su - core` to run commands as the core user. Also has the minor benefit of propagating error codes, though I opted to keep the explicit checks anyway. 2. Grep and awk calls were looking only for `fedora-toolbox-`. Simply expand the search to also accept `rhel-toolbox-`. 3. The rhel-9.8 variant is using [this toolbox](https://github.com/coreos/toolbox). This is the one that adds the most complexity to this test, as commands are completely different, the script uses root podman and requires extra configuration.

travier · 2026-03-20T15:30:40Z

tests/kola/toolbox/config.bu

+    # RHEL registry requires credentials - use quay.io registry and a Fedora image for testing
+    - path: /var/home/core/.toolboxrc
+      contents:
+        inline: |
+          REGISTRY=quay.io
+          IMAGE=fedora/fedora-toolbox:latest
+          TOOLBOX_NAME=fedora-toolbox-latest
+          AUTHFILE=/var/home/core/config.json


Unfortunately that will not test the same thing as the legacy coreos/toolbox behaves differently when there are special labels on the image and I think I remember those being there in the RHEL one but not the upstream containers/toolbox one.

Would you mind explaining, I don't really understand. Would I need to get it to run the RHEL images here then? How do I get the required authentication?

I assume this is what you're referring to with the labels

travier · 2026-03-20T15:34:44Z

As this will never be used in FCOS, I think you should keep the current FCOS test as is and duplicate it in RHCOS instead. That way we will not have to undo the change here when we move away from the legacy one in RHCOS and we'll only have to delete it there.

Rolv-Apneseth · 2026-03-20T15:59:35Z

As this will never be used in FCOS, I think you should keep the current FCOS test as is and duplicate it in RHCOS instead. That way we will not have to undo the change here when we move away from the legacy one in RHCOS and we'll only have to delete it there.

But then any changes/additions will need to be made in 2 places even though they'll eventually both be using containers/toolbox right?

For example, I was thinking we should add at least a line in this test checking toolbox works when $SHELL is unset or not exported, like that issue we had on openshift nodes running rhcos 10.2.

gemini-code-assist bot reviewed Feb 18, 2026

View reviewed changes

tests/kola/toolbox/test.sh Outdated Show resolved Hide resolved

tests/kola/toolbox/test.sh Outdated Show resolved Hide resolved

Rolv-Apneseth force-pushed the toolbox_test branch from 445a653 to 6bf3929 Compare February 18, 2026 14:00

Rolv-Apneseth force-pushed the toolbox_test branch from 6bf3929 to 1c5679e Compare March 20, 2026 10:53

dustymabe reviewed Mar 20, 2026

View reviewed changes

Rolv-Apneseth force-pushed the toolbox_test branch from 1c5679e to 4faf68a Compare March 20, 2026 14:35

Rolv-Apneseth force-pushed the toolbox_test branch from 4faf68a to 66f5ac1 Compare March 20, 2026 15:14

travier reviewed Mar 20, 2026

View reviewed changes

Conversation

Rolv-Apneseth commented Feb 18, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Rolv-Apneseth commented Feb 18, 2026

Uh oh!

Rolv-Apneseth commented Mar 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rolv-Apneseth Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

travier commented Mar 20, 2026

Uh oh!

Rolv-Apneseth commented Mar 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Rolv-Apneseth Mar 20, 2026 •

edited

Loading