Why no logicalType=binary

[Whistran]

Hi,

We would like to use ODCS to define geospatial data tables.

However these tables may contain data in WKB (Well-Known Binary), as defined by the Open Geospatial Consortium (OGC). I think it is a pity to use a logicalType="string" in this case, because it also exists the WKT (Well-known text), a string equivalent format for representing vector geometry objects.

Why not allow at least a logicalType="binary" or "blob" or "raw" ?

[pflooky]

Hi @Whistran , to give you some more insight into why we have the current set of logical types without binary, we adopted the types as described by OpenAPI. You can check this RFC for more details where we first introduced the data types.

We have expanded upon this list though due to users wanting additional support for timestamp and time as seen in this RFC.

I will bring this up as part of the monthly discussion within the TSC.

[joostvenema]

+1 for support for geospatial features at the logical level. To me it would make sense to base this on ISO 19125-1, Simple Feature Access. So, logicalType="geometry". Of course options about the spatial reference system and subclasses are also helpful.

E.g.:

- column: parcel
  logicalType: geometry
  logicalTypeOptions:
    subClass: Polygon
    spatialReferenceSystem: EPSG:4326
    dimensions: 2

[Whistran]

Thank you @pflooky for these details, I understand better now.

If I'm not wrong, currently to describe a geospatial geometry column using your standard, one must specify something like this:

properties:
      - name: footprint
        logicalType: string
        logicalTypeOptions:
          format: binary
          description: "WGS84 polygon encoded in WKB"

As @joostvenema mentioned, it would be great to integrate geospatial data directly at the logical level with specific options, modeled on the ISO 19125-1 standard. This would make your standard more easily compatible with other specification like GeoArrow, GeoParquet or new types in Iceberg v3.

[jgperrin]

@andypetrella This is in your wheelhouse, do you want to get an RFC started?

[andypetrella]

Yes, good topic for the TSC and work on a RFC.
We could check on the ISO-19115-* tho, which is closer to the OGC WRS instead of looking into the data services themselves.
A good point to start from could be the conceptual spec (I found this doc which has a data type appendix https://www.ncei.noaa.gov/sites/default/files/2020-04/ISO%2019115-2%20Workbook_Part%20II%20Extentions%20for%20imagery%20and%20Gridded%20Data.pdf).

Or we could also check on the XML implementation ISO-19139-*, as this is closer to ODCS YAML defs (well... it is an impl, soooo ^^).

This is an important topic, and also might be taken us to an area where the list of types will explode. As much as there are patterns for date (just an analogy) - for which we used logical options for example to "specify" the expected format the type takes. But, yeah, let's dip one's toe in :D

[Whistran]

@andypetrella

might be taken us to an area where the list of types will explode

Not necessarily: for example the v3 specification of the Iceberg table format which also implements this standard, only added two types: geometry and geography

Source : https://iceberg.apache.org/spec/#appendix-g-geospatial-notes

[andypetrella]

Hi, I agree that we can stick to a generic type for now, though limiting this to storage can be restrictive. While a data contract could be more specific regarding geo-column values, these details—such as restricted BBox, geotypes, or holes—might be better encoded as data quality rules. To move forward with a minimal viable specification for geo-information, I believe we need to include at least these three elements: - physicalType: Typically WKT or WKB. - logicalType: geometry or geography (this could eventually expand to include specific types like point or polygon if not handled by data quality attributes). - logicalTypeOptions: CRS (using a URN following the OGC 07-092r3 spec, e.g., urn:ogc:def:crs:EPSG::4326). A note on the physicalType, which is how data is stored. In fact, WKB is most generally used for storage, and WKT for interaction. Hence, we may never see WKT or people may provide it because that is how they interact with the data. This can be misleading because the driver or functions handle the conversion. I agree that we should move this discussion to a formal RFC entry. Best regards, Andy