You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -814,7 +814,7 @@
814
814
815
815
### Added
816
816
817
-
* Add [deploy](https://marquezproject.github.io/marquez/deployment-overview.html) docs for running Marquez on AWS [@wslulciuc](https://github.com/wslulciuc) [@merobi-hub](https://github.com/merobi-hub)
817
+
* Add [deploy](https://marquezproject.ai/deployment-overview.html) docs for running Marquez on AWS [@wslulciuc](https://github.com/wslulciuc) [@merobi-hub](https://github.com/merobi-hub)
818
818
819
819
### Changed
820
820
@@ -824,10 +824,10 @@
824
824
825
825
### Deprecated
826
826
827
-
* The following endpoints have been deprecated and are **scheduled to be removed in** `0.25.0`. Please use the [`/lineage`](https://marquezproject.github.io/marquez/openapi.html#tag/Lineage/paths/~1lineage/post) endpoint when collecting source, dataset, and job metadata [@wslulciuc](https://github.com/wslulciuc):
828
-
* [`/sources`](https://marquezproject.github.io/marquez/openapi.html#tag/Sources/paths/~1sources~1{source}/put) endpoint to collect source metadata
829
-
* [`/datasets`](https://marquezproject.github.io/marquez/openapi.html#tag/Datasets/paths/~1namespaces~1{namespace}~1datasets~1{dataset}/put) endpoint to collect dataset metadata
830
-
* [`/jobs`](https://marquezproject.github.io/marquez/openapi.html#tag/Jobs/paths/~1namespaces~1{namespace}~1jobs~1{job}/put) endpoint to collect job metadata
827
+
* The following endpoints have been deprecated and are **scheduled to be removed in** `0.25.0`. Please use the [`/lineage`](https://marquezproject.ai/openapi.html#tag/Lineage/paths/~1lineage/post) endpoint when collecting source, dataset, and job metadata [@wslulciuc](https://github.com/wslulciuc):
828
+
* [`/sources`](https://marquezproject.ai/openapi.html#tag/Sources/paths/~1sources~1{source}/put) endpoint to collect source metadata
829
+
* [`/datasets`](https://marquezproject.ai/openapi.html#tag/Datasets/paths/~1namespaces~1{namespace}~1datasets~1{dataset}/put) endpoint to collect dataset metadata
830
+
* [`/jobs`](https://marquezproject.ai/openapi.html#tag/Jobs/paths/~1namespaces~1{namespace}~1jobs~1{job}/put) endpoint to collect job metadata
831
831
832
832
### Fixed
833
833
@@ -906,7 +906,7 @@
906
906
907
907
### Deprecated
908
908
909
-
* Run API [endpoints](https://marquezproject.github.io/marquez/openapi.html#tag/Jobs) that create or modify a job run (**scheduled to be removed in** `0.19.0`). Please use the **POST** `/api/v1/lineage` [endpoint](https://marquezproject.github.io/marquez/openapi.html#tag/Lineage/paths/~1lineage/post) when collecting job run metadata. [@wslulciuc](https://github.com/wslulciuc)
909
+
* Run API [endpoints](https://marquezproject.ai/openapi.html#tag/Jobs) that create or modify a job run (**scheduled to be removed in** `0.19.0`). Please use the **POST** `/api/v1/lineage` [endpoint](https://marquezproject.ai/openapi.html#tag/Lineage/paths/~1lineage/post) when collecting job run metadata. [@wslulciuc](https://github.com/wslulciuc)
910
910
* Airflow integration, please use the [`openlineage-airflow`](https://pypi.org/project/openlineage-airflow) library instead. [@wslulciuc](https://github.com/wslulciuc)
911
911
* Spark integration, please use the [`openlineage-spark`](https://search.maven.org/artifact/io.openlineage/openlineage-spark) library instead. [@wslulciuc](https://github.com/wslulciuc)
912
912
* Write only clients for `java` and `python` (**scheduled to be removed in** `0.19.0`) [@wslulciuc](https://github.com/wslulciuc)
Copy file name to clipboardExpand all lines: README.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -64,7 +64,7 @@ Start all services:
64
64
$ sh ./docker/up.sh
65
65
```
66
66
67
-
> **Tip:** Use the `--build` flag to build images from source, and/or `--seed` to start Marquez with sample lineage metadata. For a more complete example using the sample metadata, please follow our [quickstart](https://marquezproject.github.io/marquez/quickstart.html) guide.
67
+
> **Tip:** Use the `--build` flag to build images from source, and/or `--seed` to start Marquez with sample lineage metadata. For a more complete example using the sample metadata, please follow our [quickstart](https://marquezproject.ai/quickstart.html) guide.
68
68
69
69
> **Note:** Port 5000 is now reserved for MacOS. If running locally on MacOS, you can run `./docker/up.sh --api-port 9000` to configure the API to listen on port 9000 instead. Keep in mind that you will need to update the URLs below with the appropriate port number.
70
70
@@ -78,7 +78,7 @@ You can open [http://localhost:3000](http://localhost:3000) to begin exploring t
78
78
79
79
**`HTTP API`**
80
80
81
-
The Marquez [HTTP API](https://marquezproject.github.io/marquez/openapi.html) listens on port `5000` for all calls and port `5001` for the admin interface. The admin interface exposes helpful endpoints like `/healthcheck` and `/metrics`. To verify the HTTP API server is running and listening on `localhost`, browse to [http://localhost:5001](http://localhost:5001). To begin collecting lineage metadata as OpenLineage events, use the [LineageAPI](https://marquezproject.github.io/marquez/openapi.html#tag/Lineage/paths/~1lineage/post) or an OpenLineage [integration](https://openlineage.io/docs/integrations/about).
81
+
The Marquez [HTTP API](https://marquezproject.ai/openapi.html) listens on port `5000` for all calls and port `5001` for the admin interface. The admin interface exposes helpful endpoints like `/healthcheck` and `/metrics`. To verify the HTTP API server is running and listening on `localhost`, browse to [http://localhost:5001](http://localhost:5001). To begin collecting lineage metadata as OpenLineage events, use the [LineageAPI](https://marquezproject.ai/openapi.html#tag/Lineage/paths/~1lineage/post) or an OpenLineage [integration](https://openlineage.io/docs/integrations/about).
82
82
83
83
> **Note:** By default, the HTTP API does not require any form of authentication or authorization.
84
84
@@ -90,7 +90,7 @@ To explore metadata via graphql, browse to [http://localhost:5000/graphql-playgr
90
90
91
91
We invite everyone to help us improve and keep documentation up to date. Documentation is maintained in this repository and can be found under [`docs/`](https://github.com/MarquezProject/marquez/tree/main/docs).
92
92
93
-
> **Note:** To begin collecting metadata with Marquez, follow our [quickstart](https://marquezproject.github.io/marquez/quickstart.html) guide. Below you will find the steps to get up and running from source.
93
+
> **Note:** To begin collecting metadata with Marquez, follow our [quickstart](https://marquezproject.ai/quickstart.html) guide. Below you will find the steps to get up and running from source.
94
94
95
95
## Versions and OpenLineage Compatibility
96
96
@@ -112,7 +112,7 @@ Marquez uses a _multi_-project structure and contains the following modules:
112
112
113
113
*[`api`](https://github.com/MarquezProject/marquez/tree/main/api): core API used to collect metadata
114
114
*[`web`](https://github.com/MarquezProject/marquez/tree/main/web): web UI used to view metadata
115
-
*[`clients`](https://github.com/MarquezProject/marquez/tree/main/clients): clients that implement the HTTP [API](https://marquezproject.github.io/marquez/openapi.html)
115
+
*[`clients`](https://github.com/MarquezProject/marquez/tree/main/clients): clients that implement the HTTP [API](https://marquezproject.ai/openapi.html)
> **Note:** The `integrations` module was removed in [`0.21.0`](https://github.com/MarquezProject/marquez/blob/main/CHANGELOG.md#removed), so please use an OpenLineage [integration](https://openlineage.io/integration) to collect lineage events easily.
@@ -168,7 +168,7 @@ By default, Marquez uses the following ports:
168
168
```bash
169
169
$ ./gradlew :api:runShadow
170
170
```
171
-
Marquez listens on port `8080` for all API calls and port `8081` for the admin interface. To verify the HTTP API server is running and listening on `localhost`, browse to [http://localhost:8081](http://localhost:8081). We encourage you to familiarize yourself with the [data model](https://marquezproject.github.io/marquez/#data-model) and [APIs](https://marquezproject.github.io/marquez/openapi.html) of Marquez. To run the web UI, please follow the steps outlined [here](https://github.com/MarquezProject/marquez/tree/main/web#development).
171
+
Marquez listens on port `8080` for all API calls and port `8081` for the admin interface. To verify the HTTP API server is running and listening on `localhost`, browse to [http://localhost:8081](http://localhost:8081). We encourage you to familiarize yourself with the [data model](https://marquezproject.ai/#data-model) and [APIs](https://marquezproject.ai/openapi.html) of Marquez. To run the web UI, please follow the steps outlined [here](https://github.com/MarquezProject/marquez/tree/main/web#development).
172
172
173
173
> **Note:** By default, the HTTP API does not require any form of authentication or authorization.
174
174
@@ -178,7 +178,7 @@ Marquez listens on port `8080` for all API calls and port `8081` for the admin i
The Marquez [HTTP API](https://marquezproject.github.io/marquez/openapi.html) server relies only on PostgreSQL to store dataset, job, and run metadata allowing for minimal operational overhead. We recommend a cloud provided databases, such as AWS [RDS](https://aws.amazon.com/rds/postgresql), when deploying Marquez onto Kubernetes.
12
+
The Marquez [HTTP API](https://marquezproject.ai/openapi.html) server relies only on PostgreSQL to store dataset, job, and run metadata allowing for minimal operational overhead. We recommend a cloud provided databases, such as AWS [RDS](https://aws.amazon.com/rds/postgresql), when deploying Marquez onto Kubernetes.
Copy file name to clipboardExpand all lines: docs/docs/deployment/running-on-aws.mdx
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -43,7 +43,7 @@ To create an AWS EKS cluster, please follow the steps outlined in the AWS EKS [d
43
43
44
44
## AWS RDS
45
45
46
-
Next, create an AWS RDS instance as outlined in the AWS RDS [documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CreateDBInstance.html). This database will be used to store dataset, job, and run metadata collected as [OpenLineage](https://openlineage.io) events via the Marquez [HTTP API](https://marquezproject.github.io/marquez/openapi.html).
46
+
Next, create an AWS RDS instance as outlined in the AWS RDS [documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_CreateDBInstance.html). This database will be used to store dataset, job, and run metadata collected as [OpenLineage](https://openlineage.io) events via the Marquez [HTTP API](https://marquezproject.ai/openapi.html).
[OpenLineage](https://openlineage.io) was initially prototyped using Marquez, with the [initial draft](https://github.com/OpenLineage/OpenLineage/blob/main/CHANGELOG.md#010---2021-08-12) of the spec taking inspiration from Marquez's [data model](https://lucid.app/lucidchart/f918ce01-9eb4-4900-b266-49935da271b8/view?page=8xAE.zxyknLQ#). OpenLineage events are collected via [`POST``/lineage`](https://marquezproject.github.io/marquez/openapi.html#tag/Lineage) calls, and can be queried via the `lineage_events` table using the `run_uuid` associated with the event. The _current_ schema for the `lineage_events` table is defined below:
11
+
[OpenLineage](https://openlineage.io) was initially prototyped using Marquez, with the [initial draft](https://github.com/OpenLineage/OpenLineage/blob/main/CHANGELOG.md#010---2021-08-12) of the spec taking inspiration from Marquez's [data model](https://lucid.app/lucidchart/f918ce01-9eb4-4900-b266-49935da271b8/view?page=8xAE.zxyknLQ#). OpenLineage events are collected via [`POST``/lineage`](https://marquezproject.ai/openapi.html#tag/Lineage) calls, and can be queried via the `lineage_events` table using the `run_uuid` associated with the event. The _current_ schema for the `lineage_events` table is defined below:
0 commit comments