Skip to content

Commit 8f9ec69

Browse files
committed
Docs: Refresh Python SDK examples
1 parent 2213b70 commit 8f9ec69

69 files changed

Lines changed: 1056 additions & 7228 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
/sdk-examples/ @open-metadata/ingestion @open-metadata/documentation-team
Lines changed: 31 additions & 115 deletions
Original file line numberDiff line numberDiff line change
@@ -1,139 +1,55 @@
1-
name: Test SDK Examples
1+
name: Test Python SDK examples
22

33
on:
4-
pull_request:
5-
paths:
6-
- 'sdk-examples/**'
7-
- '.github/workflows/test-sdk-examples.yml'
4+
pull_request: {}
85
push:
9-
branches:
10-
- main
11-
- 'claude/**'
12-
paths:
13-
- 'sdk-examples/**'
14-
- '.github/workflows/test-sdk-examples.yml'
6+
branches: [main]
157

16-
jobs:
17-
test:
18-
name: Test SDK Examples (Python 3.10)
19-
runs-on: ubuntu-latest
20-
permissions:
21-
contents: read
22-
pull-requests: write
23-
24-
steps:
25-
- name: Checkout code
26-
uses: actions/checkout@v4
27-
with:
28-
fetch-depth: 0
29-
30-
- name: Set up Python 3.10
31-
uses: actions/setup-python@v4
32-
with:
33-
python-version: '3.10'
34-
cache: 'pip'
35-
36-
- name: Install dependencies
37-
run: |
38-
python -m pip install --upgrade pip setuptools wheel
39-
pip install --prefer-binary -r sdk-examples/requirements-test.txt
40-
41-
- name: Run tests with pytest and coverage
42-
working-directory: sdk-examples
43-
run: |
44-
pytest tests/ -v --cov=. --cov-report=term-missing --cov-report=xml --cov-report=html
45-
46-
- name: Verify coverage data exists
47-
working-directory: sdk-examples
48-
run: |
49-
if [ ! -f .coverage ]; then
50-
echo "Error: No coverage data found. Coverage was not collected."
51-
exit 1
52-
fi
53-
echo "Coverage data verified at .coverage"
54-
55-
- name: Coverage comment
56-
uses: py-cov-action/python-coverage-comment-action@v3
57-
if: github.event_name == 'pull_request'
58-
with:
59-
GITHUB_TOKEN: ${{ github.token }}
60-
MINIMUM_GREEN: 100
61-
MINIMUM_ORANGE: 90
62-
ANNOTATE_MISSING_LINES: true
63-
ANNOTATION_TYPE: warning
64-
COVERAGE_PATH: sdk-examples
8+
permissions:
9+
contents: read
6510

66-
- name: Generate coverage report
67-
if: always()
68-
working-directory: sdk-examples
69-
run: |
70-
pip install coverage
71-
coverage report --show-missing
72-
73-
lint:
74-
name: Lint SDK Examples
11+
jobs:
12+
python:
13+
name: Python 3.12 / openmetadata-ingestion 1.13
7514
runs-on: ubuntu-latest
7615

7716
steps:
78-
- name: Checkout code
17+
- name: Check out repository
7918
uses: actions/checkout@v4
8019

81-
- name: Set up Python
82-
uses: actions/setup-python@v4
20+
- name: Set up Python 3.12
21+
uses: actions/setup-python@v5
8322
with:
84-
python-version: '3.10'
23+
python-version: "3.12"
24+
cache: pip
25+
cache-dependency-path: sdk-examples/requirements-test.txt
8526

86-
- name: Install linting tools
27+
- name: Install pinned SDK and test tools
8728
run: |
8829
python -m pip install --upgrade pip
89-
pip install ruff black isort
30+
python -m pip install -r sdk-examples/requirements-test.txt
9031
91-
- name: Check formatting with black
92-
run: |
93-
black --check sdk-examples/*.py sdk-examples/tests/*.py || true
32+
- name: Compile examples
33+
run: python -m compileall -q sdk-examples
9434

95-
- name: Check import sorting with isort
96-
run: |
97-
isort --check-only sdk-examples/*.py sdk-examples/tests/*.py || true
35+
- name: Test imports, requests, and mocked SDK calls
36+
run: pytest -q sdk-examples/tests
9837

99-
- name: Lint with ruff
100-
run: |
101-
ruff check sdk-examples/*.py sdk-examples/tests/*.py || true
38+
- name: Lint
39+
run: ruff check sdk-examples
10240

103-
validate-examples:
104-
name: Validate Example Files
41+
- name: Check formatting
42+
run: ruff format --check sdk-examples
43+
44+
secrets:
45+
name: Secret scan
10546
runs-on: ubuntu-latest
10647

10748
steps:
108-
- name: Checkout code
49+
- name: Check out repository
10950
uses: actions/checkout@v4
11051

111-
- name: Set up Python
112-
uses: actions/setup-python@v4
52+
- name: Scan for secrets
53+
uses: docker://zricethezav/gitleaks:v8.27.0
11354
with:
114-
python-version: '3.10'
115-
116-
- name: Install dependencies
117-
run: |
118-
python -m pip install --upgrade pip setuptools wheel
119-
pip install --prefer-binary -r sdk-examples/requirements-test.txt
120-
121-
- name: Validate Python syntax
122-
run: |
123-
python -m py_compile sdk-examples/*.py
124-
python -m py_compile sdk-examples/tests/*.py
125-
126-
- name: Check for common issues
127-
run: |
128-
# Check all files have proper docstrings (skip shebangs and encoding declarations)
129-
for file in sdk-examples/*.py; do
130-
if ! head -n 20 "$file" | grep -E '^("""|'"'"''"'"''"'"')' > /dev/null; then
131-
echo "Missing docstring in $file"
132-
exit 1
133-
fi
134-
done
135-
136-
- name: Verify all imports can be resolved
137-
working-directory: sdk-examples
138-
run: |
139-
python -c "import setup; import services; import entities; import metadata_ops; import lineage; import queries; import advanced"
55+
args: dir /github/workspace --redact --no-banner

README.md

Lines changed: 43 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,44 @@
1-
# OpenMetadata Demo
1+
# OpenMetadata examples
22

3-
In this repo you can find resources to try out OpenMetadata with different connectors.
3+
Runnable examples for OpenMetadata and Collate.
4+
5+
## Supported Python SDK examples
6+
7+
[`sdk-examples`](./sdk-examples) is maintained and tested against:
8+
9+
- OpenMetadata server: `1.13.x`
10+
- Python package: `openmetadata-ingestion~=1.13.0`
11+
- Public API: `metadata.sdk`
12+
- Python: `3.12`
13+
14+
Install and run the table CRUD example:
15+
16+
```bash
17+
python -m venv .venv
18+
source .venv/bin/activate
19+
python -m pip install "openmetadata-ingestion~=1.13.0"
20+
21+
export OPENMETADATA_HOST="http://localhost:8585/api"
22+
export OPENMETADATA_JWT_TOKEN="<jwt-token>"
23+
python sdk-examples/02_basic_crud.py
24+
```
25+
26+
The example expects the sample schema `sample_data.ecommerce_db.shopify`. Override
27+
`DEFAULT_SCHEMA_FQN` in the script when using another schema.
28+
29+
Documentation:
30+
31+
- [OpenMetadata Python SDK](https://docs.open-metadata.org/latest/api-reference/sdk/python)
32+
- [Collate Python SDK](https://docs.getcollate.io/sdk/python)
33+
34+
## Support status
35+
36+
| Path | Status |
37+
| --- | --- |
38+
| `sdk-examples/` | Maintained. Python checks block pull requests. |
39+
| `sdk-2.0-examples/java/` | Reference only. Not covered by Python SDK CI. |
40+
| Other demo folders | Reference or integration-specific. Check the folder README and pinned dependencies before use. |
41+
42+
Python SDK maintainers: `@open-metadata/ingestion`
43+
44+
Documentation maintainers: `@open-metadata/documentation-team`

custom-connector/custom.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,4 +21,4 @@ workflowConfig:
2121
hostPort: http://localhost:8585/api
2222
authProvider: openmetadata
2323
securityConfig:
24-
jwtToken: "eyJraWQiOiJHYjM4OWEtOWY3Ni1nZGpzLWE5MmotMDI0MmJrOTQzNTYiLCJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImlzQm90IjpmYWxzZSwiaXNzIjoib3Blbi1tZXRhZGF0YS5vcmciLCJpYXQiOjE2NjM5Mzg0NjIsImVtYWlsIjoiYWRtaW5Ab3Blbm1ldGFkYXRhLm9yZyJ9.tS8um_5DKu7HgzGBzS1VTA5uUjKWOCU0B_j08WXBiEC0mr0zNREkqVfwFDD-d24HlNEbrqioLsBuFRiwIWKc1m_ZlVQbG7P36RUxhuv2vbSp80FKyNM-Tj93FDzq91jsyNmsQhyNv_fNr3TXfzzSPjHt8Go0FMMP66weoKMgW2PbXlhVKwEuXUHyakLLzewm9UMeQaEiRzhiTMU3UkLXcKbYEJJvfNFcLwSl9W8JCO_l0Yj3ud-qt_nQYEZwqW6u5nfdQllN133iikV4fM5QZsMCnm8Rq1mvLR0y9bmJiD7fwM1tmJ791TUWqmKaTnP49U493VanKpUAfzIiOiIbhg"
24+
jwtToken: "<jwt-token>"

custom-connector/custom_proto.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,4 +22,4 @@ workflowConfig:
2222
hostPort: http://localhost:8585/api
2323
authProvider: openmetadata
2424
securityConfig:
25-
jwtToken: "eyJraWQiOiJHYjM4OWEtOWY3Ni1nZGpzLWE5MmotMDI0MmJrOTQzNTYiLCJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImlzQm90IjpmYWxzZSwiaXNzIjoib3Blbi1tZXRhZGF0YS5vcmciLCJpYXQiOjE2NjM5Mzg0NjIsImVtYWlsIjoiYWRtaW5Ab3Blbm1ldGFkYXRhLm9yZyJ9.tS8um_5DKu7HgzGBzS1VTA5uUjKWOCU0B_j08WXBiEC0mr0zNREkqVfwFDD-d24HlNEbrqioLsBuFRiwIWKc1m_ZlVQbG7P36RUxhuv2vbSp80FKyNM-Tj93FDzq91jsyNmsQhyNv_fNr3TXfzzSPjHt8Go0FMMP66weoKMgW2PbXlhVKwEuXUHyakLLzewm9UMeQaEiRzhiTMU3UkLXcKbYEJJvfNFcLwSl9W8JCO_l0Yj3ud-qt_nQYEZwqW6u5nfdQllN133iikV4fM5QZsMCnm8Rq1mvLR0y9bmJiD7fwM1tmJ791TUWqmKaTnP49U493VanKpUAfzIiOiIbhg"
25+
jwtToken: "<jwt-token>"

custom-graphql/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ workflowConfig:
7575
hostPort: "http://localhost:8585/api"
7676
authProvider: "openmetadata"
7777
securityConfig:
78-
jwtToken: "eyJraWQiOiJHYjM4OWE
78+
jwtToken: "<jwt-token>"
7979
```
8080
8181
**Important Configuration Notes:**

custom-graphql/user_updater.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,4 +19,4 @@ workflowConfig:
1919
hostPort: "http://localhost:8585/api"
2020
authProvider: "openmetadata"
2121
securityConfig:
22-
jwtToken: "eyJraWQiOiJHYjM4OWEtOWY3Ni1nZGpzLWE5MmotMDI0MmJrOTQzNTYiLCJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImlzQm90IjpmYWxzZSwiaXNzIjoib3Blbi1tZXRhZGF0YS5vcmciLCJpYXQiOjE2NjM5Mzg0NjIsImVtYWlsIjoiYWRtaW5Ab3Blbm1ldGFkYXRhLm9yZyJ9.tS8um_5DKu7HgzGBzS1VTA5uUjKWOCU0B_j08WXBiEC0mr0zNREkqVfwFDD-d24HlNEbrqioLsBuFRiwIWKc1m_ZlVQbG7P36RUxhuv2vbSp80FKyNM-Tj93FDzq91jsyNmsQhyNv_fNr3TXfzzSPjHt8Go0FMMP66weoKMgW2PbXlhVKwEuXUHyakLLzewm9UMeQaEiRzhiTMU3UkLXcKbYEJJvfNFcLwSl9W8JCO_l0Yj3ud-qt_nQYEZwqW6u5nfdQllN133iikV4fM5QZsMCnm8Rq1mvLR0y9bmJiD7fwM1tmJ791TUWqmKaTnP49U493VanKpUAfzIiOiIbhg"
22+
jwtToken: "<jwt-token>"

dynamic_csv_importer/csv_importer.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -303,7 +303,7 @@ class DebugArgs:
303303
csv_path = "sample_dbt_jaffle.csv"
304304
entities = "all" # or "Table" or "DashboardDataModel"
305305
url = "http://localhost:8585/api"
306-
jwt_token = "eyJraWQiOiJHYjM4OWEtOWY3Ni1nZGpzLWE5MmotMDI0MmJrOTQzNTYiLCJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImlzQm90IjpmYWxzZSwiaXNzIjoib3Blbi1tZXRhZGF0YS5vcmciLCJpYXQiOjE2NjM5Mzg0NjIsImVtYWlsIjoiYWRtaW5Ab3Blbm1ldGFkYXRhLm9yZyJ9.tS8um_5DKu7HgzGBzS1VTA5uUjKWOCU0B_j08WXBiEC0mr0zNREkqVfwFDD-d24HlNEbrqioLsBuFRiwIWKc1m_ZlVQbG7P36RUxhuv2vbSp80FKyNM-Tj93FDzq91jsyNmsQhyNv_fNr3TXfzzSPjHt8Go0FMMP66weoKMgW2PbXlhVKwEuXUHyakLLzewm9UMeQaEiRzhiTMU3UkLXcKbYEJJvfNFcLwSl9W8JCO_l0Yj3ud-qt_nQYEZwqW6u5nfdQllN133iikV4fM5QZsMCnm8Rq1mvLR0y9bmJiD7fwM1tmJ791TUWqmKaTnP49U493VanKpUAfzIiOiIbhg"
306+
jwt_token = "<jwt-token>"
307307
report_only = True # Set to False to actually update entities
308308

309309
args = DebugArgs()

0 commit comments

Comments
 (0)