Oauth2 mock

.github/workflows/run_integration_tests.yml

@@ -10,8 +10,14 @@ on:
         default: "latest"
     secrets:
         INTEGRATION_TEST_AZURE_CLIENT_SECRET:
-          required: true
-          description: "Azure client secret for the service principal used in integration tests with access to keyvault"
+          required: false
+          description: |
+            Azure client secret for the service principal with read access to the
+            integration-test key vault, which now holds only the MQTT credentials.
+            No longer used for authentication between the services: those tokens come
+            from a local OAuth2 mock issuer. Kept optional rather than removed because
+            this workflow is consumed at @main by every robotics repo and each one
+            declares it; removing it outright would break them all at once.
 
   workflow_dispatch:
     inputs:
@@ -25,6 +31,9 @@ permissions:
   contents: read
 
 env:
+  # Only MQTT credentials are still read from the key vault. Authentication between
+  # the services is handled by a local OAuth2 mock issuer started by the test suite,
+  # so no Entra ID app registrations are involved.
   KEYVAULT_NAME: FlotillaTestsKv
   AZURE_TENANT_ID: 3aa4a235-b6e2-48d5-9195-7fcf05b459b0
   AZURE_CLIENT_ID: 17d7c036-e4ff-4df6-87fd-0d648a36a727
@@ -53,15 +62,15 @@ jobs:
         run: |
           echo "Retrieving secrets from Key Vault: $KEYVAULT_NAME"
 
-          # You can list the secret names here
+          # MQTT authentication is username/password, validated by the broker
+          # against the hashed passwd_file committed in equinor/flotilla, so these
+          # remain real secrets. The app-registration client secrets that used to
+          # be fetched here are gone: service-to-service auth now runs against a
+          # local OAuth2 mock issuer.
           secrets=(
-            INTEGRATION-TESTS-CLIENT-SECRET
             FLOTILLA-MQTT-PASSWORD
-            FLOTILLA-AZURE-CLIENT-SECRET
             FLOTILLA-BROKER-SERVER-KEY
-            ISAR-AZURE-CLIENT-SECRET
             ISAR-MQTT-PASSWORD
-            SARA-AZURE-CLIENT-SECRET
             SARA-MQTT-PASSWORD
           )
 
@@ -120,13 +129,9 @@ jobs:
           GIT_REPOSITORY_FOR_MIGRATIONS_REF: ${{ steps.images.outputs.git_ref }}
           SARA_GIT_REPOSITORY_FOR_MIGRATIONS_REF: ${{ steps.images.outputs.git_ref }}
 
-          INTEGRATION_TESTS_CLIENT_SECRET: ${{ env.INTEGRATION_TESTS_CLIENT_SECRET }}
           FLOTILLA_MQTT_PASSWORD: ${{ env.FLOTILLA_MQTT_PASSWORD }}
-          FLOTILLA_AZURE_CLIENT_SECRET: ${{ env.FLOTILLA_AZURE_CLIENT_SECRET }}
           FLOTILLA_BROKER_SERVER_KEY: ${{ env.FLOTILLA_BROKER_SERVER_KEY }}
-          ISAR_AZURE_CLIENT_SECRET: ${{ env.ISAR_AZURE_CLIENT_SECRET }}
           ISAR_MQTT_PASSWORD: ${{ env.ISAR_MQTT_PASSWORD }}
-          SARA_AZURE_CLIENT_SECRET: ${{ env.SARA_AZURE_CLIENT_SECRET }}
           SARA_MQTT_PASSWORD: ${{ env.SARA_MQTT_PASSWORD }}
 
         run: uv run --frozen pytest -s -n auto robotics_integration_tests

README.md

@@ -11,6 +11,29 @@ The following components are currently included in the integration tests:
 - PostgreSQL database
 - Azure Blob Storage (emulated with Azurite)
 - [ISAR Robot](https://github.com/equinor/isar-robot) (your friendly neighbourhood mocked robot which provides the answers you need)
+- [SARA](https://github.com/equinor/sara) (storage and analysis of robot acquired data)
+- A local OAuth2 mock issuer (see [Authentication](#authentication))
+
+## Authentication
+The tests run with authentication **enabled and genuinely exercised**, but without Microsoft
+Entra ID. A local [oauth2-mock-server](https://github.com/axa-group/oauth2-mock-server)
+container acts as the OpenID Connect issuer for the whole stack:
+
+- Flotilla and SARA run with `ASPNETCORE_ENVIRONMENT=IntegrationTest`, which selects their
+  `appsettings.IntegrationTest.json` and points token validation at the mock.
+- ISAR is pointed at the mock with `ISAR_OPENID_CONFIG_URL`.
+- Flotilla acquires its downstream ISAR/SARA tokens from the mock too, via a
+  `GenericOidcAuthorizationHeaderProvider` registered only in that environment.
+- The test process mints its own tokens from the same issuer.
+
+This means **no app registrations, no tenant and no client secrets** are needed, and there is
+nothing to rotate. The container fixtures assert that each service rejects unauthenticated
+callers before any test runs, and the mission tests attempt unauthorised interference mid-flight
+(wrong audience, missing role) and then assert the mission completed unaffected — so a
+misconfiguration cannot silently disable authentication.
+
+MQTT is the one exception: it uses username/password validated by the broker against the
+hashed `passwd_file` committed in `equinor/flotilla`, so those credentials remain real secrets.
 
 ## Run the integration tests through remote workflow call
 To run the integration tests in a remote repository, this [workflow](./.github/workflows/run_integration_tests.yml) has been set up. 
@@ -55,6 +78,10 @@ This snippet will enable you to run the integration tests manually and automatic
 INTEGRATION_TEST_AZURE_CLIENT_SECRET
 ```
 
+This secret now only grants read access to the MQTT credentials in the key vault; it is no
+longer used for authentication between the services. It is still declared by every consuming
+repository, so it is kept for compatibility rather than removed.
+
 The input `lane` determines which image tag should be applied to the internally developed packages like Flotilla and ISAR. If input is set as `lane=dev` the newest development images (corresponding to newest push to main branch) will be used while `lane=latest` will use the newest release. 
 
 ## Local development
@@ -63,23 +90,57 @@ Clone the repository and install dependencies with [uv](https://docs.astral.sh/u
 uv sync
 ```
 
-Ensure the following secrets are populated in your local environment, either as environment variables or in a `.env` file in the repository root directory.
+Ensure the following secrets are populated in your local environment, either as environment variables or in a `.env` file in the repository root directory. These are the MQTT credentials; no
+Azure app registration secrets are needed, and you do **not** need to be logged in with `az`.
 
 ```
-INTEGRATION_TESTS_CLIENT_SECRET
-FLOTILLA_AZURE_CLIENT_SECRET
 FLOTILLA_BROKER_SERVER_KEY
 FLOTILLA_MQTT_PASSWORD
-ISAR_AZURE_CLIENT_SECRET
 ISAR_MQTT_PASSWORD
-SARA_AZURE_CLIENT_SECRET
 SARA_MQTT_PASSWORD
 ```
 
-They may all be found in the integration test [keyvault](https://portal.azure.com/#@StatoilSRM.onmicrosoft.com/resource/subscriptions/c389567b-2dd0-41fa-a5da-d86b81f80bda/resourceGroups/FlotillaIntegrationTests/providers/Microsoft.KeyVault/vaults/FlotillaTestsKv/overview).
+They may be found in the integration test [keyvault](https://portal.azure.com/#@StatoilSRM.onmicrosoft.com/resource/subscriptions/c389567b-2dd0-41fa-a5da-d86b81f80bda/resourceGroups/FlotillaIntegrationTests/providers/Microsoft.KeyVault/vaults/FlotillaTestsKv/overview).
 
 You may now run the tests with
 
 ```bash
 uv run pytest -s .
 ```
+
+### Running against locally built images
+
+By default the tests pull `ghcr.io/equinor/{flotilla-backend,sara,isar-robot}`. A change that
+spans armada *and* one of those services therefore cannot be verified until the service change
+is merged and an image published — even though the armada side is what proves the service side
+works.
+
+To close that gap, build the images from your local working copies:
+
+```bash
+scripts/build_local_images.sh           # build and verify
+scripts/build_local_images.sh --run     # ... and run the full suite against them
+```
+
+The script expects the sibling checkouts of the superrepo (`../isar`, `../isar-robot`,
+`../flotilla`, `../sara`); override with `ISAR_DIR`, `ISAR_ROBOT_DIR`, `FLOTILLA_DIR`,
+`SARA_DIR`. It verifies each image before handing back, because a subtly broken build otherwise
+shows up only as an unexplained timeout several minutes into the suite.
+
+The database schema is taken from the same local checkouts, via `FLOTILLA_MIGRATIONS_SOURCE_DIR`
+and `SARA_MIGRATIONS_SOURCE_DIR`, so application code and schema always agree. The directory is
+mounted read-only and copied into the migrations container, which means **uncommitted and
+untracked migrations are picked up**. Set either variable on its own if you want to mix a local
+schema with published images.
+
+Two things worth knowing:
+
+- **`flotilla`, `sara` and `isar` are built from the working tree**, so uncommitted changes are
+  included. **`isar-robot` is cloned**, so only committed changes are — the script warns if that
+  checkout is dirty. It has to be cloned because its Dockerfile bind-mounts `.git`, and in the
+  superrepo that is a submodule *file* rather than a directory.
+- `isar-robot`'s `uv.lock` pins `isar` from PyPI, so the locally built `isar` wheel is installed
+  over the released one.
+
+The mosquitto broker is always the published image, and the OAuth2 mock is built automatically
+by the test fixtures.

pyproject.toml

@@ -11,13 +11,10 @@ dependencies = [
   "pytest",
   "testcontainers[postgres,azurite,mqtt]",
     "azure-storage-blob",
-    "azure-identity",
-    "azure-keyvault-secrets",
     "pydantic",
     "pydantic-settings",
     "loguru",
     "python-dotenv",
-    "msal",
     "requests"
 ]
 

robotics_integration_tests/armada.py

@@ -10,20 +10,20 @@
 )
 from robotics_integration_tests.custom_containers.isar import IsarRobot
 from robotics_integration_tests.custom_containers.mosquitto import FlotillaBroker
+from robotics_integration_tests.custom_containers.oauth2_mock import OAuth2Mock
 from robotics_integration_tests.custom_containers.postgres import (
     FlotillaDatabase,
     SaraDatabase,
 )
 from robotics_integration_tests.custom_containers.teams_webhook_receiver import (
     TeamsWebhookReceiver,
 )
-from robotics_integration_tests.utilities.keyvault import Keyvault
 
 
 class Armada:
     def __init__(self) -> None:
         self.network: Network | None = None
-        self.keyvault: Keyvault | None = None
+        self.oauth_mock: OAuth2Mock | None = None
         self.test_id: str = ""
         self.flotilla_database: FlotillaDatabase | None = None
         self.flotilla_broker: FlotillaBroker | None = None