Marquez should allow a field with no datatype

[pawel-big-lebowski]

Openlineage spec allows field with no datatype:

                "type" : {
                  "description" : "The type of the field.",
                  "type" : "string",
                  "example" : "VARCHAR|INT|..."
                },

Marquez validation requires non-null fields.

[wslulciuc]

@pawel-big-lebowski, if we don't know the datatype, we have a few choices to use as defaults:

• null
• An empty string ("")
• UNKNOWN

I think null would be the most appropriate.

[pawel-big-lebowski]

I am not able to reproduce the behavior.

• I did the check code and classes: DatasetFieldMapper, DatasetFieldRowMapper and marquez.service.models.LineageEvent.SchemaField do allow null values.
• I've created a test which uses marquez.db.LineageTestUtils#createLineageRow with a dataset having fields with empty data type and no data types provided. The test passes.
• Then I manually run Jupyter and Marquez in docker and sent an event:

event = {
    'eventType': 'COMPLETE',
    'eventTime': '2022-10-25T11:35:31.341Z',
    'job': {
        'namespace': 'anothar-job-namespace',
        'name': 'another-job',
        'facets': {'documentation': None, 'sourceCodeLocation': None, 'sql': None}
    },
    'run': {
        'runId': 'ae8f3ab7-254b-4d81-9c0a-c152f6sdac90'
    },
    'inputs': [
        {
            'namespace': 'hive://metastore', # read a dataset by its logical name
            'name': 'default.some_table',
        }
    ],
    'outputs': [
         {
            'namespace': 'another-namespace', # write to some other dataset
            'name': 'another-table',
            'facets': {
                'schema': {
                    '_producer': 'https://github.com/OpenLineage/OpenLineage/tree/0.15.1/integration/spark',
                    '_schemaURL': 'https://openlineage.io/spec/1-0-3/OpenLineage.json#/$defs/RunEvent',
                    'fields': [
                        { 'name': 'a', 'type': '', 'description': '' },
                        { 'name': 'b', 'description': '' }
                    ]
                }
            }
        }
    ],
    'producer': 'https://github.com/OpenLineage/OpenLineage/tree/0.15.1/integration/spark',
    'schemaURL': 'https://openlineage.io/spec/1-0-3/OpenLineage.json#/$defs/RunEvent'
}

which also got successfully written.