Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions tests/kola/toolbox/config.bu
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
variant: fcos
version: 1.6.0
storage:
files:
# Toolbox configuration for older streams using https://github.com/coreos/toolbox
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by this exactly?

Copy link
Copy Markdown
Member Author

@Rolv-Apneseth Rolv-Apneseth Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah sorry, wasn't sure how to word it. I suppose it's only RHCOS 9.x and older that use the coreos toolbox

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about

Toolbox configuration for RHCOS variants < 10.x using https://github.com/coreos/toolbox

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok I see. So this is wholly for the legacy toolbox?

TBH in that case I would just put it in the test itself for the legacy case:

cat <<EOF > /var/home/core/.toolboxrc
REGISTRY=quay.io
IMAGE=fedora/fedora-toolbox:latest
TOOLBOX_NAME=fedora-toolbox-latest
AUTHFILE=/dev/null
EOF

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, RHCOS 10 also use the legacy coreos/toolbox implementation. So it's a RHCOS vs FCOS thing right now.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I was testing it was only 9.8 that had it, not even c9s.

Here's the toolbox package metadata from the release browser for each (not sure where else to get a list of packages):

  • 9.8: [ "toolbox", "0", "0.1.2", "1.rhaos4.22.el9", "noarch" ]
  • 10.0: [ "toolbox", "0", "0.3", "1.el10", "x86_64" ]
  • c9s: [ "toolbox", "0", "0.3", "1.el9", "x86_64" ]
  • c10s: [ "toolbox", "0", "0.3", "1.el10", "x86_64" ]

# RHEL registry requires credentials - use quay.io registry and a Fedora image for testing
- path: /var/home/core/.toolboxrc
contents:
inline: |
REGISTRY=quay.io
IMAGE=fedora/fedora-toolbox:latest
TOOLBOX_NAME=fedora-toolbox-latest
AUTHFILE=/var/home/core/config.json
Comment on lines +6 to +13
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately that will not test the same thing as the legacy coreos/toolbox behaves differently when there are special labels on the image and I think I remember those being there in the RHEL one but not the upstream containers/toolbox one.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind explaining, I don't really understand. Would I need to get it to run the RHEL images here then? How do I get the required authentication?

I assume this is what you're referring to with the labels

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind explaining, I don't really understand. Would I need to get it to run the RHEL images here then? How do I get the required authentication?

Yes unfortunately.

I assume this is what you're referring to with the labels

Yes

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But let's not block on that ,we can start with what you have here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I can work on that after. Do you have any idea as to how I can get that kind of authentication from within the test?

# Toolbox fails if the AUTHFILE does not exist, but we don't actually need credentials
- path: /var/home/core/config.json
contents:
inline: "{}"
Comment on lines +15 to +17
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand the point of this

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will double check but I believe when I was running it, the coreos toolbox would fail if the file didn't exist

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it fails without that. Either:

Error: credential file is not accessible: faccessat /var/lib/kubelet/config.json: no such file or directory

Or, if specifying AUTHFILE:

Error: credential file is not accessible: faccessat /var/home/core/config.json: no such file or directory

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so you mean coreos/toolbox fails if a credentials file doesn't exist? Does it work if you use AUTHFILE=/dev/null ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately not:

Error: unable to copy from source docker://quay.io/fedora/fedora-toolbox:latest: initializing source docker://quay.io/fedora/fedora-toolbox:latest: getting username and password: reading JSON file "/dev/null": unmarshaling JSON at "/dev/null": unexpected end of JSON input

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And yes, this whole butane file is only for coreos/toolbox

88 changes: 64 additions & 24 deletions tests/kola/toolbox/test.sh
Original file line number Diff line number Diff line change
@@ -1,52 +1,92 @@
#!/bin/bash
## kola:
## tags: "platform-independent needs-internet"
## # This test only runs on FCOS because RHCOS is missing the `machinectl` command.
## # Additionally, there are some distro specific choices made for this test that
## # should/could be adapted for RHCOS.
## # TODO-RHCOS: adapt test for RHCOS specifics or create separate RHCOS toolbox test
## distros: fcos
## # Toolbox container is currently available only for x86_64 and aarch64 in Fedora
## architectures: x86_64 aarch64
## description: Make sure that basic toolbox functionality works (creating,
## running commands, and removing).

# Important note: Commands are run indirectly via calls to `machinectl shell`
# to re-create the user environment needed for unprivileged podman
# functionality. However, machinectl shell does not propagate the exit
# code/status of the invoked shell process thus we need additionnal checks to
# ensure that previous commands were successful.

set -xeuo pipefail

# shellcheck disable=SC1091
. "$KOLA_EXT_DATA/commonlib.sh"

# IMPORTANT: Commands are run indirectly via `su - core` to re-create the
# user environment needed for unprivileged podman functionality.
run_as_core() {
su - core -c "$(printf '%q ' "$@")"
}

# Functions for testing basic functionality - overridden depending on toolbox being used
toolbox_create() {
run_as_core /bin/toolbox create --assumeyes 1> /dev/null
}

toolbox_run_basic() {
run_as_core /bin/toolbox run touch ok_toolbox
}

toolbox_list() {
run_as_core /bin/toolbox list --containers
}

toolbox_count() {
toolbox_list | grep --count -E "(fedora|rhel)-toolbox-" || true
}

toolbox_rm() {
toolbox="$(toolbox_list | awk '/(fedora|rhel)-toolbox-/ {print $2}')"
run_as_core /bin/toolbox rm -f "${toolbox}"
}

# Older variants (e.g. RHEL-9.8) use https://github.com/coreos/toolbox
# NOTE: script uses privileged podman
if file /bin/toolbox | grep -q "shell script"; then
echo "Using toolbox script <https://github.com/coreos/toolbox>"

toolbox_create() {
# Container created on first run of any command
run_as_core /bin/toolbox true
}

toolbox_run_basic() {
run_as_core /bin/toolbox touch /host/home/core/ok_toolbox
}

toolbox_list() {
/bin/podman ps -a
}

toolbox_rm() {
toolbox="$(toolbox_list | awk '/(fedora|rhel)-toolbox-/ {print $1}')"
/bin/podman rm -f "${toolbox}"
}
fi

# Try five times to create the toolbox to avoid container registry infra flakes
for i in {1..5}; do
machinectl -q shell core@ /bin/toolbox create --assumeyes 1>/dev/null
if [[ $(machinectl -qE TERM=dumb shell core@ /bin/toolbox list --containers | grep --count fedora-toolbox-) -ne 1 ]]; then
echo "Could not create toolbox on try: $i"
sleep 10
else
toolbox_create || true
if [[ $(toolbox_count) -eq 1 ]]; then
break
fi

if [[ $i -eq 5 ]]; then
fatal "Could not create toolbox after 5 attempts"
fi

echo "Could not create toolbox on try: $i"
sleep 10
done
if [[ $(machinectl -qE TERM=dumb shell core@ /bin/toolbox list --containers | grep --count fedora-toolbox-) -ne 1 ]]; then
fatal "Could not create toolbox"
fi
ok toolbox create

machinectl -q shell core@ /bin/toolbox run touch ok_toolbox
toolbox_run_basic
if [[ ! -f '/home/core/ok_toolbox' ]]; then
fatal "Could not run a simple command inside a toolbox"
fi
ok toolbox run

toolbox="$(machinectl -qE TERM=dumb shell core@ /bin/toolbox list --containers | awk '/fedora-toolbox-/ {print $2}')"
machinectl -q shell core@ /bin/podman stop "${toolbox}"
machinectl -q shell core@ /bin/toolbox rm "${toolbox}"
if [[ -n "$(machinectl -qE TERM=dumb shell core@ /bin/toolbox list --containers)" ]]; then
toolbox_rm
if [[ $(toolbox_count) -ne 0 ]]; then
fatal "Could not remove the toolbox container"
fi
ok toolbox rm