No CI lane verifies a Workbench login under SSO, and the mock-IdP lane cannot, because the test it selects skips by design whenever the provider is not password.
src/vip_tests/workbench/test_auth.py:90, inside the @given("Workbench is accessible at the configured URL") step, does:
if auth_provider != "password":
pytest.skip(f"test_auth only supports password auth, not {auth_provider!r}")mock-idp-e2e runs vip verify --idp keycloak --filter "test_connect_login_ui or test_workbench_login", so auth_provider is oidc (or saml) and test_workbench_login always skips at setup. Since pytest exits 0 when there are skips but no failures, the job reports success.
Evidence
The skip reason is identical before and after the SSO fix in #601, taken from the JUnit artifacts of two runs:
run 32089485370 (before): test_auth only supports password auth, not 'oidc'
run 32172732884 (after): test_auth only supports password auth, not 'oidc'
Run 32172732884 is the useful one: Workbench SSO now genuinely completes there (>>> Workbench authenticated. on all three legs, absent from every earlier run), the container is healthy, and the login test still skips. So this gate is independent of whether authentication works.
Why it matters
Options
- Give the module an SSO path so the scenario asserts a completed SSO session instead of driving the password form, keeping the password assertions for password deployments.
- Or point the SSO lanes at a different test that asserts an authenticated Workbench session, and stop filtering to
test_workbench_login there.
Either way the lane should not be able to report success while its only Workbench test never runs; #596 covers that exit-code contract separately.
No CI lane verifies a Workbench login under SSO, and the mock-IdP lane cannot, because the test it selects skips by design whenever the provider is not
password.src/vip_tests/workbench/test_auth.py:90, inside the@given("Workbench is accessible at the configured URL")step, does:mock-idp-e2erunsvip verify --idp keycloak --filter "test_connect_login_ui or test_workbench_login", soauth_providerisoidc(orsaml) andtest_workbench_loginalways skips at setup. Since pytest exits 0 when there are skips but no failures, the job reports success.Evidence
The skip reason is identical before and after the SSO fix in #601, taken from the JUnit artifacts of two runs:
Run 32172732884 is the useful one: Workbench SSO now genuinely completes there (
>>> Workbench authenticated.on all three legs, absent from every earlier run), the container is healthy, and the login test still skips. So this gate is independent of whether authentication works.Why it matters
mock-idp-e2everifies nothing today, andstatusgates on that job.--provider samleither — the gate skips it before any SAML-specific behaviour is exercised.Options
test_workbench_loginthere.Either way the lane should not be able to report success while its only Workbench test never runs; #596 covers that exit-code contract separately.