Add Docker-based compatibility test kit for external validation

.dockerignore

@@ -0,0 +1,5 @@
+.venv
+.git
+.jj
+.*cache
+__pycache__

.github/workflows/docker.yml

@@ -0,0 +1,56 @@
+name: Build and Push Docker Image to GHCR
+
+on:
+  push:
+    branches:
+      - 'master'
+  pull_request:
+
+jobs:
+  compat-kit:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write # Required to push packages to GHCR
+    steps:
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8
+        with:
+          persist-credentials: false
+      - uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435
+        with:
+          driver: docker-container
+      - uses: docker/login-action@184bdaa0721073962dff0199f1fb9940f07167d1
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }} # Uses the GitHub user/org name that triggered the workflow
+          password: ${{ secrets.GITHUB_TOKEN }} # Automatically provided by GitHub
+      - name: Generate image tags
+        id: meta
+        env:
+          REF: ${{ github.ref }}
+          EVENT_NAME: ${{ github.event_name }}
+          REPOSITORY: ${{ github.repository }}
+          EVENT_NUMBER: ${{ github.event.number }}
+        run: |
+          if [[ "$REF" == "refs/heads/master" ]]; then
+            echo "tags=ghcr.io/$REPOSITORY/compat-kit:latest" >> $GITHUB_OUTPUT
+            echo "push=true" >> $GITHUB_OUTPUT
+          elif [[ "$EVENT_NAME" == "pull_request" ]]; then
+            echo "tags=ghcr.io/$REPOSITORY/compat-kit:pr-$EVENT_NUMBER" >> $GITHUB_OUTPUT
+            echo "push=true" >> $GITHUB_OUTPUT
+          else
+            echo "push=false" >> $GITHUB_OUTPUT
+          fi
+          echo "created=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_OUTPUT
+      - uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83
+        with:
+          context: ./
+          file: ./compat_kit/Dockerfile
+          push: ${{ steps.meta.outputs.push }}
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: |
+            org.opencontainers.image.revision=${{ github.sha }}
+            org.opencontainers.image.source=${{ github.server_url }}/${{ github.repository }}
+            org.opencontainers.image.created=${{ steps.meta.outputs.created }}
+          cache-from: type=gha,scope=${{ github.ref_name }}-compat-kit # Cache layers to speed up builds
+          cache-to: type=gha,mode=max,scope=${{ github.ref_name }}-compat-kit # Store layers in cache for future builds

.gitignore

@@ -4,3 +4,5 @@ data.py
 vm_data.py
 /scripts/guests/windows/id_rsa.pub
 .envrc
+*.egg-info
+*.log

README.md

@@ -130,6 +130,10 @@ This command installs the dependency locally and update the `uv.lock` file.
 The lock file allows `uv` to install the dependencies at the exact same versions
 on all the environments.
 
+## Server Compatibility Testing Kit
+
+For automated testing of XCP-ng on new hardware, see the **Compatibility Test Kit** at [compat_kit/README.md](compat_kit/README.md).
+
 ## Other requirements
 * XCP-ng hosts that you can ssh to using an SSH key, non-interactively
 * VM images suited to what the tests want. Some tests want a linux VM with SSH, available to import as an OVA over HTTP, for example.

compat_kit/Dockerfile

@@ -0,0 +1,19 @@
+FROM fedora:43
+RUN dnf install -y --setopt=install_weak_deps=False \
+    curl \
+    git \
+    iputils \
+    netcat \
+    openssh-clients \
+    sshpass \
+    uv && \
+    dnf clean all
+RUN mkdir ~/.ssh && chmod 600 ~/.ssh
+WORKDIR /app
+COPY pyproject.toml uv.lock .python-version /app/
+RUN uv sync --no-dev --frozen
+ENV PATH="/app/.venv/bin:$PATH"
+COPY ./ /app/
+RUN chmod 600 compat_kit/id_ed25519*
+ENTRYPOINT ["uv", "run", "--no-dev", "./compat_kit/entrypoint.py"]
+

compat_kit/README.md

@@ -0,0 +1,83 @@
+# XCP-ng Server Compatibility Test Kit
+
+## Requirements
+
+- **Docker** installed on your machine (or Docker Desktop for macOS/Windows).
+- **Hosts to test:**
+  - At least one x86_64 server with XCP-ng installed (the release you want to test),
+    [updated with the latest updates](https://xcp-ng.org/docs/updates.html).
+  - A disk for the system, and a separate, empty disk for testing.
+  - Optionally, a second identical host. If provided, will be automatically
+    joined to the first to form a pool for storage migration and live migration
+    tests. **Both hosts must have the same SSH root password.**
+  - Internet access to download the test image.
+  - SSH access available (root user; password-based authentication).
+- **Time:** The test suite execution might take up to an hour, depending on the hardware.
+
+## Security Notes
+
+The test VM is configured with a known access key and is **strictly reserved for testing purposes only**.
+
+## Quick Start
+
+Interactive mode (recommended):
+
+```bash
+docker run -it --rm -v ${PWD}:/app/logs:Z ghcr.io/xcp-ng/xcp-ng-tests/compat-kit
+```
+
+You will be prompted for:
+
+- IP/hostname of the pool master.
+- IP/hostname of a second host (optional; press Enter to skip).
+- SSH root password (must be the same for both hosts).
+
+Alternatively, provide arguments on the command line:
+
+```bash
+docker run --rm \
+  -v ${PWD}:/app/logs \
+  ghcr.io/xcp-ng/xcp-ng-tests/compat-kit \
+  --master-host 192.168.1.10 \
+  --second-host 192.168.1.11 \
+  --password your_root_password
+```
+
+After the container completes, test log files will be in the current directory:
+
+- `test_kit_1.log` — Single-host tests.
+- `test_kit_2.log` — Two-host pool tests — only if a second host was provided.
+- `test_kit_3.log` — Xen-level tests.
+
+Open these files locally with your text editor or log viewer.
+
+## Command-line options
+
+```text
+  --master-host MASTER_HOST
+                        IP or hostname of the pool master
+  --second-host SECOND_HOST
+                        IP or hostname of the second host (optional, for pool tests)
+  --password PASSWORD   SSH root password (if not provided, will be prompted)
+  --log-dir LOG_DIR     Directory where log files will be written (default: /app/logs)
+  --log-level {debug,info,warning,error,critical}
+                        Logging level (default: info)
+  --vm-image-url VM_IMAGE_URL
+                        URL to the VM image to download (default: https://nextcloud.vates.tech/index.php/s/MmEjo8qYo7Ccs2B/download)
+  --skip-cleanup        Don't remove SSH keys from hosts on exit
+  --dry-run             Print what would be done without making any changes
+  --non-interactive     Don't prompt for missing parameters; fail if required arguments are missing
+```
+
+## Development
+
+### Building the Docker image
+
+From the xcp-ng-tests repository root:
+
+```bash
+docker build -t ghcr.io/xcp-ng/xcp-ng-tests/compat-kit -f compat_kit/Dockerfile .
+```
+
+This creates an image with all dependencies pre-installed, including the xcp-ng-tests suite and
+test runner tools.

compat_kit/__init__.py

@@ -0,0 +1 @@
+"""XCP-ng Compatibility Test Kit."""