Skip to content

Commit 56efbb8

Browse files
merobi-hubmerobi-hub
committed
Merge branch 'main' of github.com:MarquezProject/marquez into docs/add-demo
2 parents a1a0962 + 1cfa6b7 commit 56efbb8

File tree

187 files changed

+11635
-4723
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

187 files changed

+11635
-4723
lines changed

‎.circleci/api-load-test.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
set -e
1515

1616
# Build version of Marquez
17-
readonly MARQUEZ_VERSION=0.51.0-SNAPSHOT
17+
readonly MARQUEZ_VERSION=0.52.0-SNAPSHOT
1818
# Fully qualified path to marquez.jar
1919
readonly MARQUEZ_JAR="api/build/libs/marquez-api-${MARQUEZ_VERSION}.jar"
2020

‎.circleci/db-migration.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
# Version of PostgreSQL
1414
readonly POSTGRES_VERSION="14"
1515
# Version of Marquez
16-
readonly MARQUEZ_VERSION=0.50.0
16+
readonly MARQUEZ_VERSION=0.51.0
1717
# Build version of Marquez
1818
readonly MARQUEZ_BUILD_VERSION="$(git log --pretty=format:'%h' -n 1)" # SHA1
1919
readonly POSTGRES_PORT=5432

‎.env.example‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,4 @@ API_ADMIN_PORT=5001
33
WEB_PORT=3000
44
POSTGRES_PORT=5432
55
SEARCH_PORT=9200
6-
TAG=0.50.0
6+
TAG=0.51.0

‎.github/workflows/deploy-website.yml‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
name: Deploy to GitHub Pages
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
# Review gh actions docs if you want to further define triggers, paths, etc
8+
# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#on
9+
10+
jobs:
11+
build:
12+
name: Build Docusaurus
13+
runs-on: ubuntu-latest
14+
steps:
15+
- uses: actions/checkout@v4
16+
with:
17+
fetch-depth: 0
18+
- uses: actions/setup-node@v4
19+
with:
20+
node-version: 18
21+
cache: yarn
22+
cache-dependency-path: ./docs/yarn.lock
23+
24+
- name: Install dependencies
25+
run: yarn install --frozen-lockfile
26+
working-directory: ./docs
27+
- name: Build website
28+
run: yarn build
29+
working-directory: ./docs
30+
31+
- name: Upload Build Artifact
32+
uses: actions/upload-pages-artifact@v3
33+
with:
34+
path: ./docs/build # Override path to match your build output location
35+
36+
deploy:
37+
name: Deploy to GitHub Pages
38+
needs: build
39+
40+
# Grant GITHUB_TOKEN the permissions required to make a Pages deployment
41+
permissions:
42+
pages: write # to deploy to Pages
43+
id-token: write # to verify the deployment originates from an appropriate source
44+
45+
# Deploy to the github-pages environment
46+
environment:
47+
name: github-pages
48+
url: ${{ steps.deployment.outputs.page_url }}
49+
50+
runs-on: ubuntu-latest
51+
steps:
52+
- name: Deploy to GitHub Pages
53+
id: deployment
54+
uses: actions/deploy-pages@v4

‎.github/workflows/test-website.yml‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
name: Test deployment
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- main
7+
# Review gh actions docs if you want to further define triggers, paths, etc
8+
# https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#on
9+
10+
jobs:
11+
test-deploy:
12+
name: Test deployment
13+
runs-on: ubuntu-latest
14+
steps:
15+
- uses: actions/checkout@v4
16+
with:
17+
fetch-depth: 0
18+
- uses: actions/setup-node@v4
19+
with:
20+
node-version: 18
21+
cache: yarn
22+
cache-dependency-path: ./docs/yarn.lock
23+
- name: Install dependencies
24+
run: yarn install --frozen-lockfile
25+
working-directory: ./docs
26+
- name: Test build website
27+
run: yarn build
28+
working-directory: ./docs

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -178,7 +178,7 @@ Marquez listens on port `8080` for all API calls and port `8081` for the admin i
178178

179179
## Getting Involved
180180

181-
* Website: https://marquezproject.ai
181+
* Website: https://marquezproject.github.io/marquez/
182182
* Source: https://github.com/MarquezProject/marquez
183183
* Chat: [MarquezProject Slack](https://join.slack.com/t/marquezproject/shared_invite/zt-2iylxasbq-GG_zXNcJdNrhC9uUMr3B7A)
184184
* X: [@MarquezProject](https://twitter.com/MarquezProject)

‎chart/Chart.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,4 +29,4 @@ name: marquez
2929
sources:
3030
- https://github.com/MarquezProject/marquez
3131
- https://marquezproject.github.io/marquez/
32-
version: 0.50.0
32+
version: 0.51.0

‎chart/README.md‎

Lines changed: 26 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -45,28 +45,32 @@ helm delete marquez
4545

4646
### [Marquez](https://github.com/MarquezProject/marquez) **parameters**
4747

48-
| Parameter | Description | Default |
49-
|------------------------------|----------------------------------------|--------------------------|
50-
| `marquez.serviceAccount` | K8s service account for Marquez Deploy | `default` |
51-
| `marquez.replicaCount` | Number of desired replicas | `1` |
52-
| `marquez.image.registry` | Marquez image registry | `docker.io` |
53-
| `marquez.image.repository` | Marquez image repository | `marquezproject/marquez` |
54-
| `marquez.image.tag` | Marquez image tag | `0.15.0` |
55-
| `marquez.image.pullPolicy` | Image pull policy | `IfNotPresent` |
56-
| `marquez.existingSecretName` | Name of an existing secret containing db password ('marquez-db-password' key) | `nil` |
57-
| `marquez.db.host` | PostgreSQL host | `localhost` |
58-
| `marquez.db.port` | PostgreSQL port | `5432` |
59-
| `marquez.db.name` | PostgreSQL database | `marquez` |
60-
| `marquez.db.user` | PostgreSQL user | `buendia` |
61-
| `marquez.db.password` | PostgreSQL password | `macondo` |
62-
| `marquez.migrateOnStartup` | Execute Flyway migration | `true` |
63-
| `marquez.hostname` | Marquez hostname | `localhost` |
64-
| `marquez.port` | API host port | `5000` |
65-
| `marquez.adminPort` | Heath/Liveness host port | `5001` |
66-
| `marquez.resources.limits` | K8s resource limit overrides | `nil` |
67-
| `marquez.resources.requests` | K8s resource requests overrides | `nil` |
68-
| `marquez.podAnnotations` | Additional pod annotations for Marquez | `{}` |
69-
| `marquez.extraContainers` | Additional container definitions to include inside Marquez Pod | `[]` |
48+
| Parameter | Description | Default |
49+
|------------------------------|-------------------------------------------------------------------------------|--------------------------|
50+
| `marquez.serviceAccount` | K8s service account for Marquez Deploy | `default` |
51+
| `marquez.replicaCount` | Number of desired replicas | `1` |
52+
| `marquez.image.registry` | Marquez image registry | `docker.io` |
53+
| `marquez.image.repository` | Marquez image repository | `marquezproject/marquez` |
54+
| `marquez.image.tag` | Marquez image tag | `0.15.0` |
55+
| `marquez.image.pullPolicy` | Image pull policy | `IfNotPresent` |
56+
| `marquez.existingSecretName` | Name of an existing secret containing db password ('marquez-db-password' key) | `nil` |
57+
| `marquez.db.host` | PostgreSQL host | `localhost` |
58+
| `marquez.db.port` | PostgreSQL port | `5432` |
59+
| `marquez.db.name` | PostgreSQL database | `marquez` |
60+
| `marquez.db.user` | PostgreSQL user | `buendia` |
61+
| `marquez.db.password` | PostgreSQL password | `macondo` |
62+
| `marquez.dbRetention.enabled`| Enables retention policy | `false` |
63+
| `marquez.dbRetention.frequencyMins`| Apply retention policy at a frequency of every 'X' minutes | `15` |
64+
| `marquez.dbRetention.numberOfRowsPerBatch`| Maximum number of rows deleted per batch | `1000` |
65+
| `marquez.dbRetention.retentionDays`| Maximum retention days | `7` |
66+
| `marquez.migrateOnStartup` | Execute Flyway migration | `true` |
67+
| `marquez.hostname` | Marquez hostname | `localhost` |
68+
| `marquez.port` | API host port | `5000` |
69+
| `marquez.adminPort` | Heath/Liveness host port | `5001` |
70+
| `marquez.resources.limits` | K8s resource limit overrides | `nil` |
71+
| `marquez.resources.requests` | K8s resource requests overrides | `nil` |
72+
| `marquez.podAnnotations` | Additional pod annotations for Marquez | `{}` |
73+
| `marquez.extraContainers` | Additional container definitions to include inside Marquez Pod | `[]` |
7074

7175
### [Marquez Web UI](https://github.com/MarquezProject/marquez-web) **parameters**
7276

‎chart/templates/marquez/configmap.yaml‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,13 @@ data:
3333
url: jdbc:postgresql://${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}
3434
user: ${POSTGRES_USER}
3535
password: ${POSTGRES_PASSWORD}
36+
# Enables retention policy configuration for database (default: disabled)
37+
{{- if .Values.marquez.dbRetention }}
38+
dbRetention:
39+
frequencyMins: {{ .Values.marquez.dbRetention.frequencyMins }}
40+
numberOfRowsPerBatch: {{ .Values.marquez.dbRetention.numberOfRowsPerBatch }}
41+
retentionDays: {{ .Values.marquez.dbRetention.retentionDays }}
42+
{{- end }}
3643
3744
# Enables database migration on startup (default: true)
3845
migrateOnStartup: ${MIGRATE_ON_STARTUP}

‎chart/values.yaml‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ marquez:
2020
image:
2121
registry: docker.io
2222
repository: marquezproject/marquez
23-
tag: 0.50.0
23+
tag: 0.51.0
2424
pullPolicy: IfNotPresent
2525
## Name of the existing secret containing credentials for the Marquez installation.
2626
## When this is specified, it will take precedence over the values configured in the 'db' section.
@@ -34,6 +34,15 @@ marquez:
3434
name: marquez
3535
user: buendia
3636
password: macondo
37+
dbRetention:
38+
enabled: false
39+
# Apply retention policy at a frequency of every 'X' minutes (default: 15)
40+
frequencyMins: 15
41+
# Maximum number of rows deleted per batch (default: 1000)
42+
numberOfRowsPerBatch: 1000
43+
# Maximum retention days (default: 7)
44+
retentionDays: 7
45+
3746
## Indicates if Flyway database migration will execute upon deployment
3847
##
3948
migrateOnStartup: true
@@ -80,7 +89,7 @@ web:
8089
image:
8190
registry: docker.io
8291
repository: marquezproject/marquez-web
83-
tag: 0.50.0
92+
tag: 0.51.0
8493
pullPolicy: IfNotPresent
8594
## Marquez website will run on this port
8695
##

0 commit comments

Comments
 (0)