feat: accept any token type via ARTIFACTS_KEYRING_NOFUSS_TOKEN

README.md

@@ -26,18 +26,19 @@ Artifacts feed, this backend:
 
 1. **Discovers** the Azure AD tenant by making an unauthenticated request to the feed
    URL and parsing the `WWW-Authenticate` header.
-2. **Obtains a bearer token** using one of the supported auth flows (see below).
-3. For **user tokens** (Azure CLI): **exchanges** the bearer token for a narrower
-   `VssSessionToken` scoped to `vso.packaging`.
-4. For **service principal tokens** (managed identity, SP, WIF): returns the Entra
-   bearer token directly as Basic auth credentials.
-5. **Returns** the credentials to the caller.
+2. **Obtains a token** using one of the supported auth flows (see below).
+3. **Auto-detects the token type** and handles it appropriately:
+   - **Non-JWT tokens** (PATs, pre-exchanged session tokens): used directly — no exchange needed.
+   - **User tokens** (Azure CLI): exchanged for a narrower `VssSessionToken` scoped to `vso.packaging`.
+   - **Service principal tokens** (managed identity, SP, WIF): returned directly as bearer credentials.
+   - **System JWTs that fail exchange** (e.g. `$(System.AccessToken)`): gracefully fall back to direct use.
+4. **Returns** the credentials to the caller.
 
 ## Auth flows (priority order)
 
 | # | Flow | How it works |
 |---|------|-------------|
-| 1 | **Environment variable** | Reads a bearer token from `ARTIFACTS_KEYRING_NOFUSS_TOKEN` (or `VSS_NUGET_ACCESSTOKEN` as fallback). Also supports `ARTIFACTS_KEYRING_NOFUSS_TOKEN_FILE` pointing to a file, and auto-detects Docker BuildKit secrets at `/run/secrets/`. Best for CI and Docker builds. |
+| 1 | **Environment variable** | Reads a token from `ARTIFACTS_KEYRING_NOFUSS_TOKEN` (or `VSS_NUGET_ACCESSTOKEN` as fallback). Accepts any token type: bearer, PAT, or `$(System.AccessToken)`. Also supports `ARTIFACTS_KEYRING_NOFUSS_TOKEN_FILE` pointing to a file, and auto-detects Docker BuildKit secrets at `/run/secrets/`. Best for CI and Docker builds. |
 | 2 | **Azure CLI** | Runs `az account get-access-token`. Most common for local dev. |
 | 3 | **ADO auth helper** | Calls `~/ado-auth-helper` (created by the `ado-codespaces-auth` VS Code extension). Enables seamless auth in GitHub Codespaces. |
 | 4 | **Workload Identity** | Exchanges a federated token via `AZURE_CLIENT_ID` + `AZURE_FEDERATED_TOKEN_FILE` + `AZURE_TENANT_ID`. Best for GitHub Actions with `azure/login@v2`. |
@@ -84,20 +85,22 @@ export AZURE_CLIENT_SECRET=your-secret
 This requires the `azure-identity` package (included as a dependency). The service principal must have
 permissions on the Azure DevOps feed (e.g. Feed Reader).
 
-### Bearer token via environment variable
+### Token via environment variable
 
-For CI pipelines and Docker builds, pass a pre-minted bearer token:
+For CI pipelines and Docker builds, pass any valid ADO token — bearer token,
+PAT, or `$(System.AccessToken)`. The backend auto-detects the token type and
+does the right thing (exchange for session token, or use directly):
 
 ```bash
-export ARTIFACTS_KEYRING_NOFUSS_TOKEN=<bearer-token>
+export ARTIFACTS_KEYRING_NOFUSS_TOKEN=<token>
 ```
 
 For backward compatibility with existing `artifacts-keyring` CI configs,
 `VSS_NUGET_ACCESSTOKEN` is also accepted as a fallback.
 
 #### Reading tokens from files (`_FILE` convention)
 
-Set `ARTIFACTS_KEYRING_NOFUSS_TOKEN_FILE` to a path containing the bearer token.
+Set `ARTIFACTS_KEYRING_NOFUSS_TOKEN_FILE` to a path containing the token.
 This follows the Docker `_FILE` convention used by official images (postgres, mysql, etc.):
 
 ```bash
@@ -130,10 +133,13 @@ RUN --mount=type=secret,id=ARTIFACTS_KEYRING_NOFUSS_TOKEN \
 Build with:
 
 ```bash
-# Mint a short-lived ADO bearer token and pass it as a BuildKit secret
+# Local dev: mint a bearer token from Azure CLI
 export ADO_TOKEN=$(az account get-access-token \
   --resource 499b84ac-1321-427f-aa17-267ca6975798 --query accessToken -o tsv)
 
+# ADO pipeline: use $(System.AccessToken) directly
+# GitHub Actions: use token from azure/login step
+
 DOCKER_BUILDKIT=1 docker buildx build \
   --secret id=ARTIFACTS_KEYRING_NOFUSS_TOKEN,env=ADO_TOKEN \
   -t my-image .

src/artifacts_keyring_nofuss/_backend.py

@@ -45,6 +45,15 @@
 ]
 
 
+def _is_jwt(token: str) -> bool:
+    """Return True if *token* looks like a JWT (three dot-separated segments)."""
+    parts = token.split(".")
+    if len(parts) != 3:
+        return False
+    # Each segment must be non-empty
+    return all(parts)
+
+
 def _decode_jwt_claims(bearer: str) -> dict[str, str]:
     """Decode the payload of a JWT without validation. Returns {} on failure."""
     try:
@@ -312,9 +321,28 @@ def get_credential(  # noqa: PLR0911, PLR0912, C901
 
         # Try each provider; on 401 from session token exchange, continue to
         # the next provider (the rejected bearer may be stale/cached).
+        # Track the last rejected bearer so we can fall back to using it
+        # directly if all providers are exhausted (handles $(System.AccessToken)
+        # and similar system-issued JWTs that have feed access but can't be
+        # exchanged for a VssSessionToken).
+        last_rejected_bearer: str | None = None
+
         for provider, bearer in _provider.iter_tokens(chain, tenant_id):
             account = _account_from_token(bearer)
 
+            # Non-JWT tokens (PATs, pre-exchanged session tokens) cannot be
+            # exchanged — use them directly as Basic auth credentials.
+            if not _is_jwt(bearer):
+                log.debug(
+                    "token from %s is not a JWT (PAT or session token), "
+                    "using directly for %s",
+                    provider.name,
+                    _strip_userinfo(service),
+                )
+                cred = keyring.credentials.SimpleCredential("", bearer)
+                self._cache[service] = (cred, time.monotonic())
+                return cred
+
             if _is_service_principal_token(bearer):
                 if account:
                     log.debug(
@@ -330,13 +358,14 @@ def get_credential(  # noqa: PLR0911, PLR0912, C901
             try:
                 session_tok = _session_token.exchange(bearer, vsts_authority)
             except TokenRejectedError:
-                log.warning(
+                log.debug(
                     "session token exchange returned 401 for provider %s "
                     "(authenticated as %s); "
-                    "bearer token rejected, trying another provider if available.",
+                    "bearer token rejected, trying next provider.",
                     provider.name,
                     account or "unknown",
                 )
+                last_rejected_bearer = bearer
                 continue
 
             if session_tok is None:
@@ -366,7 +395,21 @@ def get_credential(  # noqa: PLR0911, PLR0912, C901
             self._cache[service] = (cred, time.monotonic())
             return cred
 
-        # All providers exhausted
+        # All providers exhausted — fall back to using the last rejected bearer
+        # directly. This handles system-issued JWTs like $(System.AccessToken)
+        # that have feed permissions but can't be exchanged.
+        if last_rejected_bearer is not None:
+            account = _account_from_token(last_rejected_bearer)
+            log.debug(
+                "exchange failed for all providers; falling back to bearer "
+                "token directly for %s (authenticated as %s)",
+                _strip_userinfo(service),
+                account or "unknown",
+            )
+            cred = keyring.credentials.SimpleCredential("bearer", last_rejected_bearer)
+            self._cache[service] = (cred, time.monotonic())
+            return cred
+
         log.warning(
             "all auth providers failed for %s "
             "(tried: %s). "

tests/test_backend.py

@@ -21,6 +21,7 @@
     _discover,
     _ensure_scheme,
     _hostname_matches,
+    _is_jwt,
     _is_service_principal_token,
     _is_supported,
     _strip_userinfo,
@@ -562,13 +563,13 @@ def test_retries_next_provider_on_401(
     @mock.patch("artifacts_keyring_nofuss._backend._session_token.exchange")
     @mock.patch("artifacts_keyring_nofuss._backend._discover")
     @mock.patch("artifacts_keyring_nofuss._backend._provider.iter_tokens")
-    def test_returns_none_when_all_rejected(
+    def test_falls_back_to_bearer_when_all_rejected(
         self,
         mock_iter: mock.MagicMock,
         mock_discover: mock.MagicMock,
         mock_exchange: mock.MagicMock,
     ) -> None:
-        """All providers' tokens are rejected — returns None."""
+        """All providers' tokens are rejected — falls back to using bearer directly."""
         mock_discover.return_value = ("tenant", "https://app.vssps.visualstudio.com")
         mock_iter.side_effect = _iter_returning(USER_JWT)
         mock_exchange.side_effect = TokenRejectedError("rejected")
@@ -577,7 +578,10 @@ def test_returns_none_when_all_rejected(
         url = "https://pkgs.dev.azure.com/org/proj/_packaging/feed/pypi/simple/"
         cred = backend.get_credential(url, None)
 
-        assert cred is None
+        # Falls back to using the bearer token directly
+        assert cred is not None
+        assert cred.username == "bearer"
+        assert cred.password == USER_JWT
 
 
 class TestDiscoverWithUserinfo:
@@ -722,6 +726,119 @@ def test_garbage_token_defaults_to_user(self) -> None:
         assert _is_service_principal_token("not-a-jwt") is False
 
 
+# ---------------------------------------------------------------------------
+# _is_jwt
+# ---------------------------------------------------------------------------
+
+
+class TestIsJwt:
+    def test_valid_jwt_structure(self) -> None:
+        assert _is_jwt(USER_JWT) is True
+        assert _is_jwt(SP_JWT) is True
+
+    def test_opaque_pat_is_not_jwt(self) -> None:
+        assert _is_jwt("my-ado-pat-token") is False
+
+    def test_two_segments_is_not_jwt(self) -> None:
+        assert _is_jwt("header.payload") is False
+
+    def test_four_segments_is_not_jwt(self) -> None:
+        assert _is_jwt("a.b.c.d") is False
+
+    def test_empty_segment_is_not_jwt(self) -> None:
+        assert _is_jwt("header..signature") is False
+
+    def test_empty_string_is_not_jwt(self) -> None:
+        assert _is_jwt("") is False
+
+
+# ---------------------------------------------------------------------------
+# Non-JWT token passthrough and exchange fallback
+# ---------------------------------------------------------------------------
+
+
+class TestNonJwtPassthrough:
+    """Non-JWT tokens (PATs, session tokens) are returned directly."""
+
+    @mock.patch("artifacts_keyring_nofuss._backend._session_token.exchange")
+    @mock.patch("artifacts_keyring_nofuss._backend._discover")
+    @mock.patch("artifacts_keyring_nofuss._backend._provider.iter_tokens")
+    def test_pat_skips_exchange(
+        self,
+        mock_iter: mock.MagicMock,
+        mock_discover: mock.MagicMock,
+        mock_exchange: mock.MagicMock,
+    ) -> None:
+        pat = "opaque-pat-string-no-dots"
+        mock_discover.return_value = ("tenant", "https://app.vssps.visualstudio.com")
+        mock_iter.side_effect = _iter_returning(pat)
+
+        backend = ArtifactsKeyringBackend()
+        url = "https://pkgs.dev.azure.com/org/proj/_packaging/feed/pypi/simple/"
+        cred = backend.get_credential(url, None)
+
+        assert cred is not None
+        assert cred.username == ""
+        assert cred.password == pat
+        mock_exchange.assert_not_called()
+
+
+class TestExchangeFallback:
+    """JWT tokens that fail exchange fall back to direct bearer use."""
+
+    @mock.patch("artifacts_keyring_nofuss._backend._session_token.exchange")
+    @mock.patch("artifacts_keyring_nofuss._backend._discover")
+    @mock.patch("artifacts_keyring_nofuss._backend._provider.iter_tokens")
+    def test_system_token_exchange_rejected_falls_back_to_bearer(
+        self,
+        mock_iter: mock.MagicMock,
+        mock_discover: mock.MagicMock,
+        mock_exchange: mock.MagicMock,
+    ) -> None:
+        """Simulates $(System.AccessToken): JWT with scp, exchange fails."""
+        system_jwt = _make_jwt({"scp": "vso.packaging", "oid": "build-svc"})
+        mock_discover.return_value = ("tenant", "https://app.vssps.visualstudio.com")
+        mock_iter.side_effect = _iter_returning(system_jwt)
+        mock_exchange.side_effect = TokenRejectedError("rejected")
+
+        backend = ArtifactsKeyringBackend()
+        url = "https://pkgs.dev.azure.com/org/proj/_packaging/feed/pypi/simple/"
+        cred = backend.get_credential(url, None)
+
+        assert cred is not None
+        assert cred.username == "bearer"
+        assert cred.password == system_jwt
+
+    @mock.patch("artifacts_keyring_nofuss._backend._session_token.exchange")
+    @mock.patch("artifacts_keyring_nofuss._backend._discover")
+    @mock.patch("artifacts_keyring_nofuss._backend._provider.iter_tokens")
+    def test_fallback_uses_last_rejected_bearer(
+        self,
+        mock_iter: mock.MagicMock,
+        mock_discover: mock.MagicMock,
+        mock_exchange: mock.MagicMock,
+    ) -> None:
+        """Multiple providers rejected — fallback uses the last one."""
+        token_a = _make_jwt({"upn": "a@example.com", "idtyp": "user"})
+        token_b = _make_jwt({"scp": "vso.packaging", "oid": "build-svc"})
+        prov_a = _FakeProvider("provider_a")
+        prov_b = _FakeProvider("provider_b")
+
+        mock_discover.return_value = ("tenant", "https://app.vssps.visualstudio.com")
+        mock_iter.side_effect = lambda *_a, **_kw: iter(
+            [(prov_a, token_a), (prov_b, token_b)]
+        )
+        mock_exchange.side_effect = TokenRejectedError("rejected")
+
+        backend = ArtifactsKeyringBackend()
+        url = "https://pkgs.dev.azure.com/org/proj/_packaging/feed/pypi/simple/"
+        cred = backend.get_credential(url, None)
+
+        assert cred is not None
+        assert cred.username == "bearer"
+        assert cred.password == token_b  # last rejected
+
+
 # ---------------------------------------------------------------------------
 # EnvVarProvider
 # ---------------------------------------------------------------------------
@@ -830,27 +947,51 @@ class TestEnvVarProviderIntegration:
 
     @mock.patch("artifacts_keyring_nofuss._backend._session_token.exchange")
     @mock.patch("artifacts_keyring_nofuss._backend._discover")
-    def test_env_var_token_used_for_exchange(
+    def test_env_var_jwt_token_used_for_exchange(
         self,
         mock_discover: mock.MagicMock,
         mock_exchange: mock.MagicMock,
     ) -> None:
         mock_discover.return_value = ("tenant", "https://app.vssps.visualstudio.com")
         mock_exchange.return_value = "my-session-token"
 
+        # Use a real JWT so it goes through the exchange path
+        env_jwt = _make_jwt({"upn": "ci@example.com", "idtyp": "user"})
+
         backend = ArtifactsKeyringBackend()
         url = "https://pkgs.dev.azure.com/org/proj/_packaging/feed/pypi/simple/"
 
-        with mock.patch.dict("os.environ", {ENV_VAR: "env-bearer-token"}):
+        with mock.patch.dict("os.environ", {ENV_VAR: env_jwt}):
             cred = backend.get_credential(url, None)
 
         assert cred is not None
         assert cred.username == "VssSessionToken"
         assert cred.password == "my-session-token"
         mock_exchange.assert_called_once_with(
-            "env-bearer-token", "https://app.vssps.visualstudio.com"
+            env_jwt, "https://app.vssps.visualstudio.com"
         )
 
+    @mock.patch("artifacts_keyring_nofuss._backend._session_token.exchange")
+    @mock.patch("artifacts_keyring_nofuss._backend._discover")
+    def test_env_var_non_jwt_token_used_directly(
+        self,
+        mock_discover: mock.MagicMock,
+        mock_exchange: mock.MagicMock,
+    ) -> None:
+        """Non-JWT tokens (PATs) skip exchange and are returned directly."""
+        mock_discover.return_value = ("tenant", "https://app.vssps.visualstudio.com")
+
+        backend = ArtifactsKeyringBackend()
+        url = "https://pkgs.dev.azure.com/org/proj/_packaging/feed/pypi/simple/"
+
+        with mock.patch.dict("os.environ", {ENV_VAR: "my-ado-pat-token"}):
+            cred = backend.get_credential(url, None)
+
+        assert cred is not None
+        assert cred.username == ""
+        assert cred.password == "my-ado-pat-token"
+        mock_exchange.assert_not_called()
+
 
 # ---------------------------------------------------------------------------
 # WorkloadIdentityProvider