Marquez Datasource facets are not displayed in Marquez UI

[dvirgiln]

Hello,

I submitted a Datasource to Marquez. This is the datasource details when I query them using the Rest API:

{
  id: {
    namespace: "s3://hbi-dns-staging",
    name: "PCHG"
  },
  type: "DB_TABLE",
  name: "PCHG",
  physicalName: "PCHG",
  createdAt: "2022-01-11T16:15:54.887Z",
  updatedAt: "2022-01-11T16:56:04.093153Z",
  namespace: "s3://hbi-dns-staging",
  sourceName: "s3://hbi-dns-staging",
  fields: [],
  tags: [],
  lastModifiedAt: null,
  description: null,
  currentVersion: "c565864d-1a66-4cff-a5d9-2e43175cbf88",
  facets: {
    dataSource: {
      uri: "s3://hbi-dns-staging/sql-runner/2022-01-11/PCHG.avro",
      name: "s3://hbi-dns-staging",
      _producer: "ip-.dir.prod.aws.hollandandbarrett.comeu-west-1.com/",
      _schemaURL: "https://openlineage.io/spec/facets/1-0-0/DatasourceDatasetFacet.json#/$defs/DatasourceDatasetFacet"
    }
  }
}

There is a facet defined, but we are not able to see that reflected in the UI:

[collado-mike]

Hi

The SchemaDatasetFacet is the one that contains the fields for the dataset schema. How did you report the event to Marquez?

[dvirgiln]

Hello @collado-mike ,

I want to have display the URI of the table. Where it is located. In this case the files are located in S3.

About the schema, it is interesting to have the schema, the different columns of the tables. I will incorporate the schema facet, but that is a different question.

This is the code we are using to publish the URI facet:

 openLineage
      .newDatasourceDatasetFacetBuilder()
      .name(name).uri(url)

[dvirgiln]

Hello @collado-mike

Any idea?

[dvirgiln]

So no information about this dataset facet appears when there is no schema, but if I include a schema in the dataset, then all the facets details appear.

[collado-mike]

Hey, sorry about the delay - can you give me a repro case? i.e., the curl (or whatever) command you used to insert the dataset and read it from the API?

[wslulciuc]

@dvirgiln: We currently don't display the datasource details in the dataset view, but I think I'd be a great idea! I've added this issue to our roadmap 👍

[Volodymyr128]

I have the same issue, at lineage view it's possible to see facets only for tables which has fields property. I suspect it isn't an expected behaviour

[phixMe]

This is now available! 🥳