Skip to content

Marquez Dataset APIs don't return outputFacets or inputFacets #1746

@collado-mike

Description

@collado-mike

Some time ago, the OpenLineage spec was changed to include outputFacets and inputFacets in the Datasets reported by a RunEvent. Several of the integrations, including the Spark integration and the GreatExpectations one, were changed to start reporting facets, such as OutputStatistics and DataQuality as either outputFacets or inputFacets.

In Marquez, the Dataset APIs return facets for Datasets and DatasetVersions, but only read the facets field of the Datasets reported from OpenLineage. This means the newer facets reported by the integrations are absent from the API response. We should update the queries to include those newer facets.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions