Skip to content

Commit fbac9b9

Browse files
authored
Store watchdog tags in storage credentials comment (#57)
## Changes With our current implementation, the watchdog deletes all credentials blindly. Since the watchdog can keep properly tagged credentials, this PR implements this tagging. ### Linked issues See https://github.com/databrickslabs/watchdog/pull/63 ### Tests Not tested --------- Co-authored-by: Eric Vergnaud <[email protected]>
1 parent 3aa8265 commit fbac9b9

File tree

2 files changed

+10
-5
lines changed

2 files changed

+10
-5
lines changed

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -928,7 +928,7 @@ def test_storage_credential(env_or_skip, make_storage_credential, make_random):
928928
)
929929
```
930930

931-
See also [`ws`](#ws-fixture).
931+
See also [`ws`](#ws-fixture), [`watchdog_remove_after`](#watchdog_remove_after-fixture).
932932

933933

934934
[[back to top](#python-testing-for-databricks)]
@@ -1137,7 +1137,7 @@ See also [`ws`](#ws-fixture).
11371137
### `watchdog_remove_after` fixture
11381138
Purge time for test objects, representing the (UTC-based) hour from which objects may be purged.
11391139

1140-
See also [`make_catalog`](#make_catalog-fixture), [`make_cluster`](#make_cluster-fixture), [`make_instance_pool`](#make_instance_pool-fixture), [`make_job`](#make_job-fixture), [`make_model`](#make_model-fixture), [`make_pipeline`](#make_pipeline-fixture), [`make_query`](#make_query-fixture), [`make_schema`](#make_schema-fixture), [`make_serving_endpoint`](#make_serving_endpoint-fixture), [`make_table`](#make_table-fixture), [`make_warehouse`](#make_warehouse-fixture), [`watchdog_purge_suffix`](#watchdog_purge_suffix-fixture).
1140+
See also [`make_catalog`](#make_catalog-fixture), [`make_cluster`](#make_cluster-fixture), [`make_instance_pool`](#make_instance_pool-fixture), [`make_job`](#make_job-fixture), [`make_model`](#make_model-fixture), [`make_pipeline`](#make_pipeline-fixture), [`make_query`](#make_query-fixture), [`make_schema`](#make_schema-fixture), [`make_serving_endpoint`](#make_serving_endpoint-fixture), [`make_storage_credential`](#make_storage_credential-fixture), [`make_table`](#make_table-fixture), [`make_warehouse`](#make_warehouse-fixture), [`watchdog_purge_suffix`](#watchdog_purge_suffix-fixture).
11411141

11421142

11431143
[[back to top](#python-testing-for-databricks)]

src/databricks/labs/pytester/fixtures/catalog.py

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import json
12
import logging
23
from collections.abc import Generator, Callable
34
from unittest.mock import Mock
@@ -386,7 +387,7 @@ def remove(udf_info: FunctionInfo):
386387

387388

388389
@fixture
389-
def make_storage_credential(ws) -> Generator[Callable[..., StorageCredentialInfo], None, None]:
390+
def make_storage_credential(ws, watchdog_remove_after) -> Generator[Callable[..., StorageCredentialInfo], None, None]:
390391
"""
391392
Create a storage credential and return its info. Remove it after the test. Returns instance of `databricks.sdk.service.catalog.StorageCredentialInfo`.
392393
@@ -419,14 +420,18 @@ def create(
419420
aws_iam_role_arn: str = "",
420421
read_only=False,
421422
) -> StorageCredentialInfo:
423+
comment = json.dumps({"RemoveAfter": watchdog_remove_after})
422424
if aws_iam_role_arn != "":
423425
storage_credential = ws.storage_credentials.create(
424-
credential_name, aws_iam_role=AwsIamRoleRequest(role_arn=aws_iam_role_arn), read_only=read_only
426+
credential_name,
427+
aws_iam_role=AwsIamRoleRequest(role_arn=aws_iam_role_arn),
428+
read_only=read_only,
429+
comment=comment,
425430
)
426431
else:
427432
azure_service_principal = AzureServicePrincipal(directory_id, application_id, client_secret)
428433
storage_credential = ws.storage_credentials.create(
429-
credential_name, azure_service_principal=azure_service_principal, read_only=read_only
434+
credential_name, azure_service_principal=azure_service_principal, read_only=read_only, comment=comment
430435
)
431436
return storage_credential
432437

0 commit comments

Comments
 (0)