[FSTORE-2036] PR 4a/4 — Python SDK support for Unity Catalog OAuth M2M

[jimdowling]

Summary

PR 4 of 4 for FSTORE-2099, hopsworks-api half. Extends UnityCatalogConnector so the Python SDK round-trips the new OAuth fields the backend (#3032 / #3033) and frontend (logicalclocks/hopsworks-front#1919) added. Legacy PAT-only construction keeps working unchanged.

Companion loadtest PR: logicalclocks/loadtest (link in this thread once the loadtest PR is opened).

Spec: uc-oauth2/uc-oauth2.md in the per-feature workspace. Ticket: https://hopsworks.atlassian.net/browse/FSTORE-2099

What changes

• Constructor gains auth_method, client_id, client_secret, oauth_endpoint, account_id, account_host, has_access_token, has_client_secret. auth_method defaults to "PAT" when absent so existing code paths (and downstream fixtures) keep producing PAT connectors. OAUTH_M2M without an explicit oauth_endpoint defaults to "WORKSPACE", matching the frontend default.
• Write-only-friendly booleans: has_access_token and has_client_secret come from the server (hasAccessToken / hasClientSecret in camelCase). When a caller builds a connector locally with a secret in hand, the booleans fall back to "is the secret non-None" so client code that constructs in-process still reports correct state.
• from_response_json is unchanged — it already uses humps.decamelize + **kwargs splat, which picks up the new fields by name once they're declared on the constructor.
• Existing get_unity_catalog fixture updated to match the post-PR-1 backend wire format (hasAccessToken: true; no decrypted access_token in GET responses). Two new fixtures (get_unity_catalog_oauth_workspace, get_unity_catalog_oauth_account) cover the OAuth modes.
• Tests extended from 4 to 8 in TestUnityCatalogConnector — round-trip for both OAuth modes; legacy construction defaulting to PAT; OAuth construction defaulting oauth_endpoint to WORKSPACE.

Test plan

• uv run pytest python/tests/test_storage_connector.py::TestUnityCatalogConnector — 8/8 passing.
• uv run ruff check — clean.
• uv run docsig python/hsfs/storage_connector.py — clean.

🤖 Generated with Claude Code

[github-actions]

Coverage report

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
python/hsfs
feature_group.py					5226-5227, 5362
storage_connector.py					3581, 3606-3610, 3617-3625, 3633, 3648-3742, 3754-3759, 3793-3798, 3804, 3810, 3816, 3822, 3828, 3834, 3849-3851, 3870-3872, 3876-3886
python/hsfs/core
storage_connector_api.py					104-114
Project Total

This report was generated by python-coverage-comment-action

[Copilot]

Pull request overview

Extends the Python SDK’s UnityCatalogConnector model to support Unity Catalog OAuth M2M by round-tripping new OAuth-related fields and adding “secret present” booleans that reflect backend write-only secrets, while keeping legacy PAT-only construction working unchanged.

Changes:

• Add OAuth M2M fields (auth_method, client_id, client_secret, oauth_endpoint, account_id, account_host) and write-only-friendly flags (has_access_token, has_client_secret) to UnityCatalogConnector.
• Update UC connector fixtures to reflect backend write-only secret behavior and add OAuth workspace/account fixtures.
• Expand TestUnityCatalogConnector to cover PAT back-compat defaults and OAuth endpoint defaults + parsing.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
python/hsfs/storage_connector.py	Adds UC OAuth M2M fields + secret-presence booleans and exposes them via public properties.
python/tests/test_storage_connector.py	Updates UC parsing assertions (write-only token) and adds OAuth + defaulting tests.
python/tests/fixtures/storage_connector_fixtures.json	Updates UC GET fixture to new wire format and adds OAuth workspace/account fixtures.