[FSTORE-2036] PR 4a/4 — Python SDK support for Unity Catalog OAuth M2M#966
Draft
jimdowling wants to merge 1 commit into
Draft
[FSTORE-2036] PR 4a/4 — Python SDK support for Unity Catalog OAuth M2M#966jimdowling wants to merge 1 commit into
jimdowling wants to merge 1 commit into
Conversation
https://hopsworks.atlassian.net/browse/FSTORE-2036 PR 4 of 4 for FSTORE-2036, hopsworks-api half. Extend UnityCatalogConnector so the Python SDK round-trips the new OAuth fields the backend (PR 1 / PR 2) and frontend (PR 3) added. Legacy PAT-only construction keeps working unchanged. Constructor gains auth_method, client_id, client_secret, oauth_endpoint, account_id, account_host, has_access_token, and has_client_secret. auth_method defaults to "PAT" when absent so existing code paths and fixtures that construct connectors with just access_token keep producing PAT connectors. When the caller asks for OAUTH_M2M without specifying oauth_endpoint, it defaults to "WORKSPACE", matching the frontend default. has_access_token and has_client_secret are write-only-friendly booleans: the server emits them on read so a caller can tell whether a secret is on file without ever seeing it. When constructed locally with a secret in hand, has_* falls back to "is the secret non-None" so client code that builds a connector in-process still reports the correct state. from_response_json keeps using humps.decamelize + **kwargs splat; the new fields are picked up by name. The existing get_unity_catalog fixture is updated to match the post-PR-1 backend wire format (hasAccessToken: true on read; no decrypted access_token in the response). Two new fixtures (get_unity_catalog_oauth_workspace, get_unity_catalog_oauth_account) cover the OAuth modes. Tests extended from 4 to 8 in TestUnityCatalogConnector. New cases: from_response_json for OAuth workspace and OAuth account modes; legacy construction defaulting to PAT (no auth_method supplied); OAUTH_M2M construction defaulting oauth_endpoint to WORKSPACE. uv run pytest TestUnityCatalogConnector — 8/8 passing. uv run ruff check / docsig — clean. Signed-off-by: Jim Dowling <jim@hopsworks.ai> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
3 tasks
Coverage report
Click to see where and how coverage changed
This report was generated by python-coverage-comment-action |
||||||||||||||||||||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR 4 of 4 for FSTORE-2099, hopsworks-api half. Extends
UnityCatalogConnectorso the Python SDK round-trips the new OAuth fields the backend (#3032 / #3033) and frontend (logicalclocks/hopsworks-front#1919) added. Legacy PAT-only construction keeps working unchanged.Companion loadtest PR: logicalclocks/loadtest (link in this thread once the loadtest PR is opened).
Spec:
uc-oauth2/uc-oauth2.mdin the per-feature workspace. Ticket: https://hopsworks.atlassian.net/browse/FSTORE-2099What changes
auth_method,client_id,client_secret,oauth_endpoint,account_id,account_host,has_access_token,has_client_secret.auth_methoddefaults to"PAT"when absent so existing code paths (and downstream fixtures) keep producing PAT connectors.OAUTH_M2Mwithout an explicitoauth_endpointdefaults to"WORKSPACE", matching the frontend default.has_access_tokenandhas_client_secretcome from the server (hasAccessToken/hasClientSecretin camelCase). When a caller builds a connector locally with a secret in hand, the booleans fall back to "is the secret non-None" so client code that constructs in-process still reports correct state.from_response_jsonis unchanged — it already useshumps.decamelize+**kwargssplat, which picks up the new fields by name once they're declared on the constructor.get_unity_catalogfixture updated to match the post-PR-1 backend wire format (hasAccessToken: true; no decryptedaccess_tokenin GET responses). Two new fixtures (get_unity_catalog_oauth_workspace,get_unity_catalog_oauth_account) cover the OAuth modes.TestUnityCatalogConnector— round-trip for both OAuth modes; legacy construction defaulting to PAT; OAuth construction defaultingoauth_endpointtoWORKSPACE.Test plan
uv run pytest python/tests/test_storage_connector.py::TestUnityCatalogConnector— 8/8 passing.uv run ruff check— clean.uv run docsig python/hsfs/storage_connector.py— clean.🤖 Generated with Claude Code