Add a Docker Compose script to run the browser tests

Author: sbp

atr/docs/running-and-creating-tests.html

@@ -9,15 +9,23 @@ <h1 id="running-and-creating-tests">3.7. Running and creating tests</h1>
 </ul>
 <h2 id="running-tests">Running tests</h2>
 <p>We currently only have end-to-end browser tests, but we plan to expand these as part of <a href="https://github.com/apache/tooling-trusted-releases/issues/209">Issue #209</a>. Meanwhile, these browser tests serve as a simple consistency check when developing ATR.</p>
-<p>To run the tests, you will need Docker. Other OCI runtimes should work, but you will need to edit the <a href="/ref/Makefile"><code>Makefile</code></a> or run your own command. The simple way to run the tests is:</p>
+<p>To run the tests, you will need Docker. Other OCI runtimes should work, but you will need to edit the test scripts accordingly.</p>
+<h3 id="using-docker-compose">Using Docker Compose</h3>
+<p>The simplest way to run the tests is using Docker Compose, which starts both ATR and the Playwright test container:</p>
+<pre><code class="language-shell">sh tests/run-tests.sh
+</code></pre>
+<p>This uses <a href="/ref/tests/docker-compose.yml"><code>tests/docker-compose.yml</code></a> to orchestrate the test environment. The ATR server runs in one container and the Playwright tests run in another, connected via a Docker network.</p>
+<h3 id="using-host-networking">Using host networking</h3>
+<p>If you already have ATR running locally with <code>make serve-local</code>, you can run the Playwright tests directly against it instead of using Docker Compose:</p>
 <pre><code class="language-shell">make build-playwright &amp;&amp; make run-playwright
 </code></pre>
 <p>Where the two <code>make</code> invocations correspond to:</p>
 <pre><code class="language-shell">docker build -t atr-playwright -f tests/Dockerfile.playwright playwright
 docker run --net=host -it atr-playwright python3 test.py --skip-slow
 </code></pre>
-<p>In other words, we build <a href="/ref/tests/Dockerfile.playwright"><code>tests/Dockerfile.playwright</code></a>, and then run <a href="/ref/playwright/test.py"><code>playwright/test.py</code></a> inside that container. The container is called <code>atr-playwright</code>; if you want to give the container a different name, then you'll need to run the manual <code>docker</code> commands. Replace <code>docker</code> with the name of your Docker-compatible OCI runtime to use an alternative runtime.</p>
-<p>The tests should, as of 14 Oct 2025, take about 20 to 25 seconds to run. The last line should be <code>Tests finished successfully</code>, and if the tests do not complete successfully there should be an obvious Python backtrace.</p>
+<p>In other words, we build <a href="/ref/tests/Dockerfile.playwright"><code>tests/Dockerfile.playwright</code></a>, and then run <a href="/ref/playwright/test.py"><code>playwright/test.py</code></a> inside that container using host networking to access your locally running ATR instance. Replace <code>docker</code> with the name of your Docker-compatible OCI runtime to use an alternative runtime.</p>
+<h3 id="test-duration">Test duration</h3>
+<p>The tests should, as of 14 Oct 2025, take about 40 to 50 seconds to run in Docker Compose, and 20 to 25 seconds to run on the host. The last line of the test output should be <code>Tests finished successfully</code>, and if the tests do not complete successfully there should be an obvious Python backtrace.</p>
 <h2 id="creating-tests">Creating tests</h2>
 <p>You can add tests to <code>playwright/test.py</code>. If you're feeling particularly adventurous, you can add separate unit tests etc., but it's okay to add tests only to the Playwright test script until <a href="https://github.com/apache/tooling-trusted-releases/issues/209">Issue #209</a> is resolved.</p>
 <h3 id="how-the-tests-work">How the tests work</h3>

atr/docs/running-and-creating-tests.md

@@ -15,7 +15,21 @@
 
 We currently only have end-to-end browser tests, but we plan to expand these as part of [Issue #209](https://github.com/apache/tooling-trusted-releases/issues/209). Meanwhile, these browser tests serve as a simple consistency check when developing ATR.
 
-To run the tests, you will need Docker. Other OCI runtimes should work, but you will need to edit the [`Makefile`](/ref/Makefile) or run your own command. The simple way to run the tests is:
+To run the tests, you will need Docker. Other OCI runtimes should work, but you will need to edit the test scripts accordingly.
+
+### Using Docker Compose
+
+The simplest way to run the tests is using Docker Compose, which starts both ATR and the Playwright test container:
+
+```shell
+sh tests/run-tests.sh
+```
+
+This uses [`tests/docker-compose.yml`](/ref/tests/docker-compose.yml) to orchestrate the test environment. The ATR server runs in one container and the Playwright tests run in another, connected via a Docker network.
+
+### Using host networking
+
+If you already have ATR running locally with `make serve-local`, you can run the Playwright tests directly against it instead of using Docker Compose:
 
 ```shell
 make build-playwright && make run-playwright
@@ -28,9 +42,11 @@ docker build -t atr-playwright -f tests/Dockerfile.playwright playwright
 docker run --net=host -it atr-playwright python3 test.py --skip-slow
 ```
 
-In other words, we build [`tests/Dockerfile.playwright`](/ref/tests/Dockerfile.playwright), and then run [`playwright/test.py`](/ref/playwright/test.py) inside that container. The container is called `atr-playwright`; if you want to give the container a different name, then you'll need to run the manual `docker` commands. Replace `docker` with the name of your Docker-compatible OCI runtime to use an alternative runtime.
+In other words, we build [`tests/Dockerfile.playwright`](/ref/tests/Dockerfile.playwright), and then run [`playwright/test.py`](/ref/playwright/test.py) inside that container using host networking to access your locally running ATR instance. Replace `docker` with the name of your Docker-compatible OCI runtime to use an alternative runtime.
+
+### Test duration
 
-The tests should, as of 14 Oct 2025, take about 20 to 25 seconds to run. The last line should be `Tests finished successfully`, and if the tests do not complete successfully there should be an obvious Python backtrace.
+The tests should, as of 14 Oct 2025, take about 40 to 50 seconds to run in Docker Compose, and 20 to 25 seconds to run on the host. The last line of the test output should be `Tests finished successfully`, and if the tests do not complete successfully there should be an obvious Python backtrace.
 
 ## Creating tests
 

playwright/test.py

@@ -34,6 +34,7 @@
 import playwright.sync_api as sync_api
 import rich.logging
 
+ATR_BASE_URL: Final[str] = os.environ.get("ATR_BASE_URL", "https://localhost.apache.org:8080")
 OPENPGP_TEST_UID: Final[str] = "<apache-tooling@example.invalid>"
 SSH_KEY_COMMENT: Final[str] = "atr-playwright-test@127.0.0.1"
 SSH_KEY_PATH: Final[str] = "/root/.ssh/id_ed25519"
@@ -92,7 +93,7 @@ def get_default_gateway_ip() -> str | None:
 
 
 def go_to_path(page: sync_api.Page, path: str, wait: bool = True) -> None:
-    page.goto(f"https://localhost.apache.org:8080{path}")
+    page.goto(f"{ATR_BASE_URL}{path}")
     if wait:
         wait_for_path(page, path)
 
@@ -313,21 +314,22 @@ def main() -> None:
     )
 
     logging.debug(f"Log level set to {args.log.upper()}")
-    # Add localhost.apache.org to /etc/hosts
-    default_gateway_ip = get_default_gateway_ip()
-    if default_gateway_ip is not None:
-        with open("/etc/hosts", "a") as f:
-            f.write(f"{default_gateway_ip} localhost.apache.org\n")
-        logging.info(f"Added localhost.apache.org to /etc/hosts with IP {default_gateway_ip}")
-    else:
-        logging.warning("Could not determine default gateway IP, skipping /etc/hosts modification")
+    if "ATR_BASE_URL" not in os.environ:
+        # Add localhost.apache.org to /etc/hosts
+        default_gateway_ip = get_default_gateway_ip()
+        if default_gateway_ip is not None:
+            with open("/etc/hosts", "a") as f:
+                f.write(f"{default_gateway_ip} localhost.apache.org\n")
+            logging.info(f"Added localhost.apache.org to /etc/hosts with IP {default_gateway_ip}")
+        else:
+            logging.warning("Could not determine default gateway IP, skipping /etc/hosts modification")
 
     run_tests(args.skip_slow, args.tidy)
 
 
 def poll_for_tasks_completion(page: sync_api.Page, project_name: str, version_name: str, revision: str) -> None:
     rev_path = f"{project_name}/{version_name}/{revision}"
-    polling_url = f"https://localhost.apache.org:8080/admin/ongoing-tasks/{rev_path}"
+    polling_url = f"{ATR_BASE_URL}/admin/ongoing-tasks/{rev_path}"
     logging.info(f"Polling URL: {polling_url}")
 
     max_wait_seconds = 18
@@ -930,17 +932,17 @@ def test_projects_02_check_directory(page: sync_api.Page, credentials: Credentia
     go_to_path(page, "/projects")
     logging.info("Project directory page loaded")
 
-    logging.info("Checking for the Apache Tooling project card")
-    h3_locator = page.get_by_text("Apache Tooling", exact=True)
-    tooling_card_locator = h3_locator.locator("xpath=ancestor::div[contains(@class, 'project-card')]")
-    sync_api.expect(tooling_card_locator).to_be_visible()
-    logging.info("Apache Tooling project card found successfully")
+    logging.info("Checking for the Apache Test project card")
+    h3_locator = page.get_by_text("Apache Test", exact=True)
+    test_card_locator = h3_locator.locator("xpath=ancestor::div[contains(@class, 'project-card')]")
+    sync_api.expect(test_card_locator).to_be_visible()
+    logging.info("Apache Test project card found successfully")
 
 
 def test_projects_03_add_project(page: sync_api.Page, credentials: Credentials) -> None:
-    base_project_label = "tooling"
-    project_name = "Apache Tooling Test Example"
-    project_label = "tooling-test-example"
+    base_project_label = "test"
+    project_name = "Apache Test Example"
+    project_label = "test-example"
 
     logging.info("Navigating to the add derived project page")
     go_to_path(page, f"/project/add/{base_project_label}")
@@ -1040,14 +1042,18 @@ def test_ssh_02_rsync_upload(page: sync_api.Page, credentials: Credentials) -> N
 
     logging.info(f"Starting rsync upload test for {project_name}-{version_name}")
 
-    gateway_ip = get_default_gateway_ip()
-    if not gateway_ip:
-        raise RuntimeError("Cannot proceed without gateway IP")
+    if "ATR_BASE_URL" in os.environ:
+        ssh_host = os.environ.get("ATR_BASE_URL", "").replace("https://", "").replace(":8080", "")
+    else:
+        gateway_ip = get_default_gateway_ip()
+        if not gateway_ip:
+            raise RuntimeError("Cannot proceed without gateway IP")
+        ssh_host = gateway_ip
 
     username = credentials.username
     ssh_command = "ssh -p 2222 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null"
     source_path = f"{source_dir_abs}/"
-    destination = f"{username}@{gateway_ip}:/{project_name}/{version_name}/"
+    destination = f"{username}@{ssh_host}:/{project_name}/{version_name}/"
 
     rsync_cmd = [
         "rsync",

tests/docker-compose.yml

@@ -0,0 +1,36 @@
+services:
+  atr:
+    build:
+      context: ..
+      dockerfile: Dockerfile.alpine
+    environment:
+      - ALLOW_TESTS=1
+      - APP_HOST=atr:8080
+      - BIND=0.0.0.0:8080
+      - SECRET_KEY=insecure-test-key
+      - SSH_HOST=0.0.0.0
+    networks:
+      - test-network
+    healthcheck:
+      test: ["CMD", "curl", "-k", "-f", "https://localhost:8080/"]
+      interval: 2s
+      timeout: 1s
+      retries: 30
+      start_period: 10s
+
+  playwright:
+    build:
+      context: ../playwright
+      dockerfile: ../tests/Dockerfile.playwright
+    depends_on:
+      atr:
+        condition: service_healthy
+    networks:
+      - test-network
+    command: python3 test.py --skip-slow
+    environment:
+      - ATR_BASE_URL=https://atr:8080
+
+networks:
+  test-network:
+    driver: bridge

tests/run-tests.sh

@@ -0,0 +1,10 @@
+#!/bin/sh
+set -eu
+
+cd "$(dirname "$0")"
+
+echo "Building and running ATR integration tests..."
+docker compose up --build --abort-on-container-exit --exit-code-from playwright
+
+# Clean up
+docker compose down -v