`bootstrap-compose` minimal profile cannot register services on a fresh install (401, missing dynamic-service-keys seed)

[who1s]

Summary

On a fresh minimal profile install of 4.7.x, every service_* container in bootstrap-compose.yaml exits with a 401 from service_server and service_hot in Elasticsearch stays empty. The platform comes up but has no services to dispatch to.

Affected versions

Observed on AL_VERSION=4.7.4.stable0 against cccs/assemblyline-rust:4.7.4.stable0. Likely affects every 4.7.x tag since the Rust service_server and the dynamic-service-keys auth model became the default.

Reproduction

1. Clone assemblyline-docker-compose.
2. Use the default .env (or any .env where SERVICE_API_KEY is set).
3. COMPOSE_PROFILES=minimal docker compose up -d
4. docker compose -f bootstrap-compose.yaml up -d
5. Wait for service_* containers to exit.

Expected: services are registered and visible in GET /service_hot/_count and in the UI.

Actual: every service_* container exits 0, but service_hot count stays at 0. service_server logs:

ERROR ... Service attempted to register without write permissions: <Name>/<version>
ERROR ... error handling /api/v1/service/register (... ms) [401 Unauthorized]
       The operation requested required permissions this api key does not have

Root cause

The Rust service_server has two auth paths in assemblyline-server/src/service_api/helpers/auth.rs:

1. Static SERVICE_API_KEY env var — always bound to allow_registry_writing: false. No fallthrough.
2. Redis hash dynamic-service-keys — entries can have allow_registry_writing: true.

Only path (2) reaches register_service (which actually writes to ES); path (1) falls into compare_service_info, which 401s when the service does not already exist — i.e. on first boot.

In production the Python updater (assemblyline_core/updater/run_updater.py) generates per-install UUIDs and writes them to dynamic-service-keys before launching REGISTER_ONLY=true containers.

The bootstrap-compose.yaml flow short-circuits this: registration containers are launched directly with SERVICE_API_KEY: ${SERVICE_API_KEY} inherited from common/service.yaml. That env var matches the service_server's static key, so every request lands on path (1) and 401s.

config/bootstrap.py currently only sets up the admin user — it does not seed dynamic-service-keys, so there is no working write-capable key anywhere in the system.

Proposed fix

Three small edits in assemblyline-docker-compose:

1. .env — introduce SERVICE_REGISTRATION_KEY (distinct from SERVICE_API_KEY).

2. common/service.yaml — change the register_service template so registration containers send the new key:

   environment:
     SERVICE_API_KEY: ${SERVICE_REGISTRATION_KEY}

3. config/bootstrap.py — after admin-user setup, write the registration key into the dynamic-service-keys Redis hash with allow_registry_writing: true. Sketch:

   from assemblyline.common import isotime
   from assemblyline.common.constants import SERVICE_API_KEY_HASH
   from assemblyline.remote.datatypes import get_client
   from assemblyline.remote.datatypes.hash import Hash
   
   key = os.environ["SERVICE_REGISTRATION_KEY"]
   assert key != os.environ.get("SERVICE_API_KEY")
   config = forge.get_config()
   redis = get_client(config.core.redis.persistent.host,
                      config.core.redis.persistent.port, False)
   Hash(SERVICE_API_KEY_HASH, redis).add(key, {
       "key": key,
       "allow_registry_writing": True,
       "expiry": isotime.now_as_iso(10 * 365 * 24 * 3600),
   })

Confirmed working locally, all 45 registration containers exit 0 and service_hot populates as expected.

Workaround

Apply the three edits above by hand on the deployment host, then docker compose -f bootstrap-compose.yaml run --rm first_time_setup followed by docker compose -f bootstrap-compose.yaml up -d.

[cccs-douglass]

Thanks for letting us know! Went about fixing it by merging the service auto installing into the core so its not duplicated between the docker and kubernetes deployments.