An example of a bundle that creates and uses a secret scope

knowledge_base/job_read_secret/README.md

@@ -0,0 +1,31 @@
+# Databricks job that reads a secret from a secret scope
+
+This example demonstrates how to define a secret scope and a job with a task that reads from it in a Databricks Asset Bundle.
+
+It includes and deploys an example secret scope, and a job with a task in a bundle that reads a secret from the secret scope to a Databricks workspace.
+
+For more information about Databricks secrets, see the [documentation](https://docs.databricks.com/aws/en/security/secrets).
+
+## Prerequisites
+
+* Databricks CLI v0.252.0 or above
+
+## Usage
+
+Modify `databricks.yml`:
+* Update the `host` field under `workspace` to the Databricks workspace to deploy to
+
+Run `databricks bundle deploy` to deploy the bundle.
+
+Run a script to write a secret to the secret scope:
+
+```
+SECRET_SCOPE_NAME=$(databricks bundle summary -o json | jq -r '.resources.secret_scopes.my_secret_scope.name')
+
+databricks secrets put-secret ${SECRET_SCOPE_NAME} example-key --string-value example-value --profile ${DATABRICKS_PROFILE}
+```
+
+Run the job:
+```
+databricks bundle run example_python_job
+```

knowledge_base/job_read_secret/databricks.yml

@@ -0,0 +1,47 @@
+bundle:
+  name: job-read-secret-example
+
+# workspace:
+#   host: https://myworkspace.cloud.databricks.com
+
+resources:
+  secret_scopes:
+    my_secret_scope:
+      name: secrets-scope-1
+      permissions:
+        - level: CAN_VIEW
+          group_name: users
+        - level: CAN_MANAGE
+          group_name: admins
+  jobs:
+    example_python_job:
+      name: "example-python-job"
+      parameters:
+        - name: "scope_name"
+          default: ${resources.secret_scopes.my_secret_scope.name}
+      tasks:
+        - task_key: example_python_task
+          spark_python_task:
+            python_file: "src/example_spark_python_task.py"
+            parameters:
+              - --scope_name={{job.parameters.scope_name}}
+
+# Defines the targets for this bundle.
+# Targets allow you to deploy the same bundle to different Databricks workspaces.
+targets:
+  prod: {
+    # No overrides
+  }
+  dev:
+    # This target is for development purposes.
+    # It defaults to the current Databricks workspace.
+    default: true
+    mode: development
+    resources:
+      secret_scopes:
+        my_secret_scope:
+          name: ${workspace.current_user.short_name}-my-secrets
+      jobs:
+        example_python_job:
+          name: "${workspace.current_user.short_name}-example-python-job"
+

knowledge_base/job_read_secret/src/example_spark_python_task.py

@@ -0,0 +1,33 @@
+#!/usr/bin/env python
+
+import os
+from datetime import datetime
+import argparse
+
+
+def main():
+    # Get current timestamp
+    now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
+
+    # Print job information
+    print(f"Example Python job started at: {now}")
+
+    # Read a secret from a passed secret scope
+    try:
+        parser = argparse.ArgumentParser()
+        parser.add_argument("-s", "--scope_name", help="Name of the secret scope")
+        args = parser.parse_args()
+        scope_name = args.scope_name
+
+        secret_value = dbutils.secrets.get(scope=scope_name, key="example-key")
+        print(
+            f"Successfully retrieved secret. First few characters: {secret_value[:3]}***"
+        )
+    except Exception as e:
+        print(f"Could not access secret: {str(e)}")
+
+    print("Example Python job completed successfully")
+
+
+if __name__ == "__main__":
+    main()