You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/automations/snowflake-metadata-sync.md
+50-18Lines changed: 50 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
import FeatureAvailability from '@site/src/components/FeatureAvailability';
2
2
3
-
# Snowflake Tag Propagation Automation
3
+
# Snowflake Metadata Sync Automation
4
4
5
5
<FeatureAvailabilitysaasOnly />
6
6
@@ -12,14 +12,16 @@ This feature is currently in Public Beta in DataHub Cloud. Reach out to your Dat
12
12
13
13
## Introduction
14
14
15
-
Snowflake Tag Propagation is an automation that allows you to sync DataHub Glossary Termsand Tags on
15
+
Snowflake Metadata Sync is an automation that allows you to sync DataHub Glossary Terms, Tags, and Descriptions on
16
16
both columns and tables back to Snowflake. This automation is available in DataHub Cloud only.
17
17
18
18
## Capabilities
19
19
20
20
- Automatically Add DataHub Glossary Terms to Snowflake Tables and Columns
21
21
- Automatically Add DataHub Tags to Snowflake Tables and Columns
22
+
- Automatically Sync DataHub Descriptions to Snowflake Tables and Columns as Comments
22
23
- Automatically Remove DataHub Glossary Terms and Tags from Snowflake Tables and Columns when they are removed in DataHub
24
+
- Support for both Username/Password and Private Key authentication
23
25
24
26
## Prerequisites
25
27
@@ -28,7 +30,8 @@ both columns and tables back to Snowflake. This automation is available in DataH
28
30
-`CREATE TAG`: Required to create new tags in Snowflake.
29
31
Ensure the user or role has this privilege on the specific schema or database where tags will be created.
30
32
-`APPLY TAG`: Required to assign tags to Snowflake objects such as tables, columns, or other database objects.
31
-
This permission must be granted at the database, schema, or object level depending on the scope.
33
+
**This permission must be granted at the ACCOUNT level** - it cannot be granted on individual schemas, tables, or views.
34
+
-`OWNERSHIP` on objects: Required to apply tags to tables and columns, as well as to update comments/descriptions. This is the most comprehensive permission and is necessary for tag operations.
32
35
33
36
### Permissions Required for Object Access
34
37
@@ -40,40 +43,61 @@ both columns and tables back to Snowflake. This automation is available in DataH
40
43
To grant the necessary permissions for a specific role (DATAHUB_AUTOMATION_ROLE), you can use the following SQL commands:
41
44
42
45
```sql
46
+
-- Database and schema access
47
+
GRANT USAGE ON DATABASE your_database TO ROLE DATAHUB_AUTOMATION_ROLE;
48
+
GRANT USAGE ON SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
49
+
43
50
-- Tag management permissions
44
51
GRANT CREATE TAG ON SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
45
-
GRANT APPLY TAG ON SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
52
+
-- APPLY TAG must be granted at ACCOUNT level
53
+
GRANT APPLY TAG ON ACCOUNT TO ROLE DATAHUB_AUTOMATION_ROLE;
46
54
47
-
-- Object access for metadata operations
48
-
GRANT USAGE ON DATABASE your_database TO ROLE DATAHUB_AUTOMATION_ROLE;
49
-
GRANT USAGE ON SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
55
+
-- Object access and modification permissions
50
56
GRANTSELECTON ALL TABLES IN SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
57
+
GRANTSELECTON ALL VIEWS IN SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
58
+
59
+
-- OWNERSHIP is required for applying tags and updating comments/descriptions
60
+
GRANT OWNERSHIP ON ALL TABLES IN SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
61
+
GRANT OWNERSHIP ON ALL VIEWS IN SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
51
62
52
-
-- Future privileges for tagging
63
+
-- Future privileges for new objects
53
64
GRANTSELECTON FUTURE TABLES IN SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
54
-
GRANT APPLY TAG ON FUTURE TABLES IN SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
65
+
GRANTSELECTON FUTURE VIEWS IN SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
66
+
GRANT OWNERSHIP ON FUTURE TABLES IN SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
67
+
GRANT OWNERSHIP ON FUTURE VIEWS IN SCHEMA your_database.your_schema TO ROLE DATAHUB_AUTOMATION_ROLE;
68
+
-- Note: APPLY TAG is granted at ACCOUNT level above and applies to all objects
55
69
```
56
70
57
-
## Enabling Snowflake Tag Sync
71
+
## Enabling Snowflake Metadata Sync
58
72
59
73
1.**Navigate to Automations**: Click on 'Govern' > 'Automations' in the navigation bar.
3.**Configure Automation**: Fill in the required fields to connect to Snowflake, along with the name, description, and category.
72
-
Note that you can limit propagation based on specific Tags and Glossary Terms. If none are selected, then ALL Tags or Glossary Terms will be automatically
73
-
propagated to Snowflake tables and columns. Finally, click 'Save and Run' to start the automation
86
+
87
+
**Authentication Options:**
88
+
89
+
-**Username/Password**: Traditional authentication using Snowflake username and password
90
+
-**Private Key**: Key pair authentication using RSA private key (more secure for automated processes)
91
+
92
+
**Sync Options:**
93
+
94
+
-**Tags & Terms**: You can limit propagation based on specific Tags and Glossary Terms. If none are selected, then ALL Tags or Glossary Terms will be automatically propagated to Snowflake tables and columns.
95
+
-**Descriptions**: Enable description sync to automatically update Snowflake table and column comments with DataHub descriptions.
96
+
97
+
Finally, click 'Save and Run' to start the automation
@@ -102,10 +126,18 @@ The back-filling of tags will be available in a future release.
102
126
103
127
:::
104
128
105
-
## Viewing Propagated Tags
129
+
## Viewing Synced Metadata
130
+
131
+
You can view propagated Tags, Terms, and updated Comments (and corresponding DataHub URNs) inside the Snowflake UI to confirm the automation is working as expected.
132
+
133
+
### Tags and Terms
134
+
135
+
Tags and glossary terms will appear as Snowflake tags on your tables and columns:
136
+
137
+
### Descriptions
106
138
107
-
You can view propagated Tags (and corresponding DataHub URNs) inside the Snowflake UI to confirm the automation is working as expected.
139
+
DataHub descriptions will be synced as Snowflake comments on tables and columns, visible in the Snowflake UI and accessible via `SHOW TABLES` and `DESCRIBE TABLE` commands.
0 commit comments