feat(cli): improve browser login instructions and error handling for Keycloak authentication

Author: iLoveTux

README.md

@@ -15,6 +15,7 @@ Currently using [OpenBao](https://openbao.org) and [OpenTofu](https://opentofu.o
 | [OpenTofu](https://opentofu.org/docs/intro/install/) | 1.8.0 | `tofu` must be on `$PATH` |
 | [Podman](https://podman.io/docs/installation) | 4.0 | `podman` must be on `$PATH` |
 | [podman-compose](https://github.com/containers/podman-compose) | 1.0 | `podman compose` plugin or `podman-compose` |
+| [Python](https://python.org) | 3.12+ | `python` must be on `$PATH` | 
 | Linux kernel | — | `IPC_LOCK` for mlock; set `disable_mlock = true` if unavailable (some WSL2 setups) |
 
 The Vault/OpenBao CLI is **not** required on the host — `tofu output` prints ready-to-run `init` and `unseal` commands. If you do have the CLI installed, set `VAULT_CACERT` to the generated CA path.
@@ -560,7 +561,7 @@ Navigate to `https://127.0.0.1:8444/admin` and log in with your admin credential
 
 1. With the `vault` client open, click the **Client scopes** tab
 2. Click the link named **`vault-dedicated`** (the first row, type will show as *Dedicated*)
-3. Click **Add mapper** → **By configuration**
+3. Click **Add mapper** → **Configure a New Mapper**
 4. Click **Group Membership**
 5. Set **Name** to `groups`
 6. Set **Token Claim Name** to `groups`
@@ -599,7 +600,7 @@ Navigate to `https://127.0.0.1:8444/admin` and log in with your admin credential
 
 1. Click the **Client scopes** tab
 2. Click **`agent-cli-dedicated`**
-3. Click **Add mapper** → **By configuration**
+3. Click **Add mapper** → **Configure a new Mapper**
 4. Click **Group Membership**
 5. Set **Name** to `groups`
 6. Set **Token Claim Name** to `groups`
@@ -610,7 +611,55 @@ Navigate to `https://127.0.0.1:8444/admin` and log in with your admin credential
 
 ---
 
-Once this is done you can verify the mapper is working before touching OpenBao — log in as your operator user and decode the resulting access token at `https://jwt.io` to confirm the `groups` claim contains `vault-operators`.
+Once this is done, verify the mapper before touching OpenBao (using the correct client token):
+
+1. In Keycloak admin, open **Clients** -> **vault** -> **Client scopes** -> **Evaluate**
+2. Select user **operator** and click **Generated access token**
+3. Keycloak shows the decoded JSON payload directly — confirm it includes `"groups": ["vault-operators"]`
+4. Repeat for **Clients** -> **agent-cli** -> **Client scopes** -> **Evaluate**, selecting the same **operator** user
+
+If `groups` is still missing, troubleshoot in this order:
+
+1. **Check user membership in the correct realm**
+    - Where: **Realm: armory** -> **Users** -> **operator** -> **Groups**
+    - Success: `vault-operators` is listed.
+    - Failure: No `vault-operators` membership.
+    - Meaning: Mapper is fine, but there is no source group data to emit.
+
+2. **Check mapper location for each client**
+    - Where: **Clients** -> **vault** -> **Client scopes** -> **vault-dedicated** -> **Mappers** and **Clients** -> **agent-cli** -> **Client scopes** -> **agent-cli-dedicated** -> **Mappers**
+    - Success: Group Membership mapper exists on both dedicated scopes.
+    - Failure: Mapper exists only on one client, wrong scope, or at realm level only.
+    - Meaning: Claim may appear for one client but not the other, or not appear at all.
+
+3. **Check mapper fields (exact values)**
+    - Expected: **Token Claim Name**=`groups`, **Add to access token**=ON, **Full group path**=OFF.
+    - Success: Values match exactly.
+    - Failure: Claim name differs (`group`, `Groups`, etc.), access token toggle OFF, or path format unexpected.
+    - Meaning: Vault expects `groups`; name/path mismatches can make the claim appear missing or unusable.
+
+4. **Regenerate token from Evaluate (do not reuse old token)**
+    - Success: New token now contains `groups`.
+    - Failure: Still missing in freshly generated token.
+    - Meaning: Not a stale-session issue; configuration is still not effective.
+
+5. **Clear active user sessions and test again**
+    - Where: **Realm: armory** -> **Sessions** -> logout user sessions, then log in again.
+    - Success: Claim appears after fresh login.
+    - Failure: No change.
+    - Meaning: If success, old session/token cache was the cause. If failure, continue below.
+
+6. **Validate OpenBao OIDC role expects the same claim name**
+    - Command: `podman exec -e VAULT_TOKEN=<ROOT_TOKEN> armory-vault bao read auth/oidc/role/operator`
+    - Success: Output shows `groups_claim` set to `groups`.
+    - Failure: `groups_claim` is different or missing.
+    - Meaning: Keycloak may emit `groups`, but OpenBao will ignore it unless claim names match.
+
+7. **Run one isolation test (temporary)**
+    - Action: Add the same Group Membership mapper at realm scope, generate token again, then remove it after testing.
+    - Success: Claim appears only with realm-level mapper.
+    - Failure: Claim still absent.
+    - Meaning: If success, dedicated scope attachment is the issue. If failure, check user/group assignment and mapper type again.
 ---
 
 ### Phase 8 — Deploy the agentic layer
@@ -623,6 +672,7 @@ Once this is done you can verify the mapper is working before touching OpenBao 
 cd vault-config/
 export TF_VAR_vault_token=<ROOT_TOKEN>
 tofu apply -var agent_enabled=true -var database_roles_enabled=true
+cd ..
 ```
 
 This command ensures both the agent AppRole and the database roles are enabled in Vault. Both variables must be set to `true` when deploying the agentic layer.
@@ -635,6 +685,7 @@ cp example.tfvars terraform.tfvars
 export TF_VAR_vault_token=<ROOT_TOKEN>
 tofu init
 tofu apply
+cd ../..
 ```
 
 This writes `role_id` and `wrapped_secret_id` to `/opt/armory/agent/approle/`.
@@ -647,7 +698,7 @@ python3 -m venv .venv
 .venv/bin/pip install -r requirements.txt
 
 export VAULT_ADDR=https://127.0.0.1:8200
-export ARMORY_CACERT=/opt/armory/vault/tls/ca.crt
+export ARMORY_CACERT=/home/vagrant/project/project-armory/vault/ca-bundle.pem
 export APPROLE_DIR=/opt/armory/agent/approle
 export KEYCLOAK_URL=https://127.0.0.1:8444
 export OIDC_CLIENT_ID=agent-cli
@@ -668,6 +719,13 @@ cd services/agent/agent/
 PKCE, and submits the task in one step. The response includes a `request_id` for
 correlating the API log with the Vault audit log.
 
+> **Headless / no-GUI environments:** `cli.py` always prints the full authorization URL
+> to stderr before attempting to open a browser. If no browser is available (e.g. a VM
+> with no desktop), copy-paste the printed URL into a browser on your host machine.
+> The callback listener runs on `http://127.0.0.1:18080/callback` — as long as port
+> 18080 is not forwarded to the host, the redirect will return to the VM's loopback
+> interface and the CLI will complete the exchange automatically once you log in.
+
 > **Same env vars:** `cli.py` reads `KEYCLOAK_URL`, `ARMORY_CACERT`, `OIDC_CLIENT_ID`,
 > and `AGENT_API_URL` from the environment — the same variables set for the API above.
 

services/agent/agent/cli.py

@@ -112,8 +112,16 @@ def log_message(self, *args):
         "code_challenge_method": "S256",
     })
 
-    print("Opening browser for Keycloak login...", file=sys.stderr)
-    webbrowser.open(auth_url)
+    print("\nOpen the following URL in your browser to log in:\n", file=sys.stderr)
+    print(f"  {auth_url}\n", file=sys.stderr)
+    try:
+        if webbrowser.open(auth_url):
+            print("(Browser opened automatically.)", file=sys.stderr)
+        else:
+            print("(Could not open a browser — paste the URL above manually.)", file=sys.stderr)
+    except Exception:
+        print("(Could not open a browser — paste the URL above manually.)", file=sys.stderr)
+    print("Waiting for login callback on http://127.0.0.1:18080/callback ...\n", file=sys.stderr)
 
     thread.join(timeout=120)