Versioned-Features Proposal: Feedback and alignment

[fmigneault]

Following are some items I identified while reviewing https://github.com/opengeospatial/ogcapi-features/blob/master/proposals/versioned-features.md

Providing general feedback and recommendations to open the floor for more design considerations.

Validity/Transaction Time Semantic

In Collection metadata, the validity-time and transaction-time are proposed.

However, it is fairly common for created/updated fields to be used to reflect when the "transaction" portion was applied. For example, STAC Common Metadata - Date and Time and OGC API Processes - Job Status use them.

For the sake of alignment, I would recommend reusing them.

Similarly, a mutable field could be used matching what OGC API Processes - DRU does.

IMO, a validity-time seems redundant, because the updated feature's datetime field (or possibly start_datetime/end_datetime in STAC) combined with the Collection's temporal extent with OGC API Common: Part 2 + UAD) should already indicate when it applies (when it is valid).

A "versioning-aware" and a "regular" feature should essentially be the same, if not undistinguishable, for a client that doesn't know the semantics of this extension.

Deprecation and Versioning link relations

I strongly recommend having a look at https://github.com/stac-extensions/version. It introduces a deprecated flag, the version field, and employs the same IANA versioning link relations proposed in this extension. It is a natural alignment to consider.

In Access to a feature and its versions, I would personally avoid using canonical, since it could be referring to something else (e.g.: a remote authority reference). Instead, I would recommend using the current link relation to reflect the preferred "latest" feature.

The first and last to reflect versioning links should take into consider other usages like paging. Possibly, these are not the same as versioning references (at least not directly). Paging could be relevant if there are a lot of updated feature revisions that could be accessed with /collections/{collectionId}/items/{featureId}/versions?limit=10&page=2 for example. The prev/next would apply as normal paging links in that context. For the others, it is tricky because a rel: start exists, but there is no rel: end. I am not quite sure what would be the best way to distinguish first/start and last/??? between versioning/paging concepts and RFC 7089 Memento definitions.

Relationship with datetime parameter - no versioning parameters?

I agree with Relationship to the existing datetime parameter and Backwards compatibility that datetime should not change its meaning and simply interact with the "temporal-validity" concept of the features.

However, the lack of a parameter to obtain the older versions can be problematic. For example, if a version used to be valid within a certain time period, and was updated to reflect a change (e.g.: a store that changed owner/name), the older feature would suddenly disappear. There are 2 workarounds.

1. Provide "duplicate features", like currently (without the extension), which will be filtered the normal way by "valid" datetime. Essentially, something similar to versions-as-features and versions-as-features-unique-ids profiles.
2. Provide some sort of ?revisions= / ?version= parameters to allow retrieving older versions of the feature.

In case (2), using /collctions/xx/items?revisions=true could allow returning all revisions of a feature (with additional filters applicable) rather than only the latest one (default behaviour). Similarly, /collctions/xx/items/{featureId}?version=x.y.z would select the specific older revision, matching the version field in the feature (or the latest if omitted).

One important distinction should be that versioned features can employ the same featureId, and therefore a version would also be required to distinguish them. If a distinct featureId is used to encode the different versions (which is also valid, but not the scope of this extension), then the multiple feature revisions would be returned in the full feature listing anyway, and datetime queries could already filter them, making the extra versioning somewhat unnecessary/redundant. Therefore, some parameters must be provided to indicate or facilitate the listing of multiple feature revisions, or within a certain "interval of validity". I believe combining ?revisions=true&datetime=../.. would allow to do this since all valid features within that time interval would be considered, not just the latest revisions.

Profiles

Personally, I think features-with-time-slices overcomplicates things and should be dropped altogether. I think there are more chances that it confuses users.

If a group wants to manage these versioned features separately and all at once, they could simply create a separate collection from them, or employ the versioning endpoint (see below).

Extra Media-Types

The /collections/{collectionId}/items/{featureId}/versions endpoint could leverage the fact that it would be a collection of features, so it could reuse the same feature-collection media-types employed by other endpoints.

[cportele]

Thanks for the super-fast and comprehensive feedback, @fmigneault!

I add comments inline:

Validity/Transaction Time Semantic

In Collection metadata, the validity-time and transaction-time are proposed.

However, it is fairly common for created/updated fields to be used to reflect when the "transaction" portion was applied. For example, STAC Common Metadata - Date and Time and OGC API Processes - Job Status use them.

For the sake of alignment, I would recommend reusing them.

Similarly, a mutable field could be used matching what OGC API Processes - DRU does.

IMO, a validity-time seems redundant, because the updated feature's datetime field (or possibly start_datetime/end_datetime in STAC) combined with the Collection's temporal extent with OGC API Common: Part 2 + UAD) should already indicate when it applies (when it is valid).

Such an approach may work for Processes, STAC or Records, but it does not work in general and in particular not for Features.

OGC API Features will never mandate any specific property names. Property names are specified by an application schema or, if no schema exists, by the API author. In addition, property names like datetime, created or updated do not work in other languages like German.

The specific roles that certain properties have are declared in the schema, if one is provided. See the x-ogc-role annotation in Schemas. For example, the properties that specify the start and end timestamp of a version are identified by the primary temporal information.

Also whether a property can or cannot be set by a mutation request is declared in the schema.

The additional information in the collection metadata provide hints about the semantics, which goes beyond the logical schema. That information is not essential, so if it is not considered helpful, the information in the Collection resource could simply be dropped.

A "versioning-aware" and a "regular" feature should essentially be the same, if not undistinguishable, for a client that doesn't know the semantics of this extension.

Which is the case.

Deprecation and Versioning link relations

I strongly recommend having a look at https://github.com/stac-extensions/version. It introduces a deprecated flag, the version field, and employs the same IANA versioning link relations proposed in this extension. It is a natural alignment to consider.

I had looked at the STAC "version" extension. My conclusion was that that extension is about something else. There may be some overlap, but it seems to be a different use case; the extension is not very specific. There is no discussion about URIs, identifiers of the resource and versions, etc. The extension is probably about some simpler kind of versioning.

Versioned features is about features where each feature is represented as a temporally continuous sequence of versions. Clients that do not know anything about the extension should interact with a feature in a meaningful way, and client that are aware of the extension should be able to access a particular version of a feature.

The Memento RFC is a close and - in my view - quite elegant match for this use case and the OGC API design, in particular pattern 1.2, which is why I used that as the starting point.

Since I do see how someone could use both extensions (STAC and OGC API) on the same resource in a meaningful way, I do not think there is a conflict.

In Access to a feature and its versions, I would personally avoid using canonical, since it could be referring to something else (e.g.: a remote authority reference). Instead, I would recommend using the current link relation to reflect the preferred "latest" feature.

The use of canonical is correct based on the definition of the link relation "designates the preferred version of a resource".

On the other hand, current would not be correct in a deprecated version / memento.

That said, the canonical link is not essential and I only added it for clarity. We could also drop it from the proposal (implementations can still add it, if they like).

The first and last to reflect versioning links should take into consider other usages like paging. Possibly, these are not the same as versioning references (at least not directly).

Link relation types are intentionally generic in their definition and first, last, prev, next are no different, they apply to all cases of a series, whether it is a sequence of pages or versions.

RFC 7089 identifies those as well as the link relations from RFC 5289 as options:

The Memento framework defines the "original", "timegate", "timemap",
and "memento" Relation Types to convey typed links among Original
Resources, TimeGates, Mementos, and TimeMaps. [...]
In addition, existing Relation Types may be
used, for example, to support navigating among Mementos. Examples
are "first", "last", "prev", "next", "predecessor-version", and
"successor-version" as detailed in RFC5988 and RFC5829.

I included both variants, but it is probably better to use only one set.

Paging could be relevant if there are a lot of updated feature revisions that could be accessed with /collections/{collectionId}/items/{featureId}/versions?limit=10&page=2 for example.

Yes, I decided to not define paging on the TimeMap resource for now, but it could be added and as you point out, at least we need a next link in that case.

The prev/next would apply as normal paging links in that context. For the others, it is tricky because a rel: start exists, but there is no rel: end. I am not quite sure what would be the best way to distinguish first/start and last/??? between versioning/paging concepts and RFC 7089 Memento definitions.

Since both are a series of resources (versions and pages), it is up to us how we make use of the link relations. The link relation gives a hint about the semantics, it will never tell the complete story.

Since the ones from RFC 5289 are explicitly for versions, we should use those for the series of versions and first, last, prev, next for pages. I will update the proposal accordingly.

Relationship with datetime parameter - no versioning parameters?

I agree with Relationship to the existing datetime parameter and Backwards compatibility that datetime should not change its meaning and simply interact with the "temporal-validity" concept of the features.

However, the lack of a parameter to obtain the older versions can be problematic. For example, if a version used to be valid within a certain time period, and was updated to reflect a change (e.g.: a store that changed owner/name), the older feature would suddenly disappear. There are 2 workarounds.

1. Provide "duplicate features", like currently (without the extension), which will be filtered the normal way by "valid" datetime. Essentially, something similar to versions-as-features and versions-as-features-unique-ids profiles.
2. Provide some sort of ?revisions= / ?version= parameters to allow retrieving older versions of the feature.

I do not understand this discussion. Most of the proposal discusses how to access particular versions of a feature. See Access to a feature and its versions and the related examples. For example, GET /api/collections/building/items/1?datetime=2005-01-01T00:00:00Z provides access to the version of feature "1" on January 1st, 2005.

In case (2), using /collctions/xx/items?revisions=true could allow returning all revisions of a feature (with additional filters applicable) rather than only the latest one (default behaviour).

This is covered by / the same as /api/collections/building/items?datetime=../..

Similarly, /collctions/xx/items/{featureId}?version=x.y.z would select the specific older revision, matching the version field in the feature (or the latest if omitted).

See above, the datetime parameter is used for this (RFC 7089 uses the Accept-Datetime header, but I think we should not require content negotiation mechanisms beyond those defined by HTTP).

One important distinction should be that versioned features can employ the same featureId, and therefore a version would also be required to distinguish them. If a distinct featureId is used to encode the different versions (which is also valid, but not the scope of this extension), then the multiple feature revisions would be returned in the full feature listing anyway, and datetime queries could already filter them, making the extra versioning somewhat unnecessary/redundant. Therefore, some parameters must be provided to indicate or facilitate the listing of multiple feature revisions, or within a certain "interval of validity". I believe combining ?revisions=true&datetime=../.. would allow to do this since all valid features within that time interval would be considered, not just the latest revisions.

The version identifier of a feature version is the datetime timestamp at the start of the version interval.

I am not sure I understand what you are trying to say, but it is part of the design that you only get a single version of a feature back when you access the Feature resource at /api/collections/building/items/{featureId} - with or without the datetime parameter.

Most feature formats are not designed to encode different states of a feature at the same time. And where that is possible, this representation is typically complex / hard to process and it will differ from the representation of a single version - which again makes it very hard to consume in clients.

Encoding a feature as a feature collection of its versions also does not work in most encodings. For example, it does not work with GeoJSON and it would break backwards compatibility.

So, it feels natural to always provide only a single version in representations of a feature.

To fetch multiple versions of a feature in a single request, the Features (items) resource can be used, with a datetime parameter that is an interval and an ids parameter to select the feature - or a CQL2 filter.

Profiles

Personally, I think features-with-time-slices overcomplicates things and should be dropped altogether. I think there are more chances that it confuses users.

I would be happy to drop the profile, if there is consensus about this (we do not plan to implement it), but I expect that there will be use cases where this could be the preferred representation - for the same reason why some application schemas used GML dynamic features in the past.

Extra Media-Types

The /collections/{collectionId}/items/{featureId}/versions endpoint could leverage the fact that it would be a collection of features, so it could reuse the same feature-collection media-types employed by other endpoints.

The versions resource is "just" the TimeMap with links to all the mementos/versions, not the versions themselves.

[fmigneault]

@cportele
I see, good insight. I did not consider the other languages. That does indeed change the usage of properties.

I think the STAC version extension is exactly the same thing in spirit, but indeed not very explicit about the URIs to employ (maybe by design to allow flexibility). I agree the Memento RFC is very good to describe most of the necessary parts in details, and there's most probably some tooling available since some website archives most realistically rely on it.

---

The use of canonical is correct based on the definition of the link relation "designates the preferred version of a resource".
On the other hand, current would not be correct in a deprecated version / memento.

Maybe this is my own interpretation, but I think the "preferred version" in this case is somewhat of an overloaded English terminology. It should probably be worded as "preferred reference" or "preferred resource" to avoid the "version" ambiguity. I believe current better reflects (semantically by the term itself) the actual "versioning" and through its definition "Refers to a resource containing the most recent item(s) in a collection of resources.".

If a deprecated/memento version was accessed, I think it would report itself as self, or item if in a listing. I do not see in which context an older version would be referenced from somewhere else beside within a version-history listing, where each feature would be an item or similar?

My main concern about this canonical/current distinction is that there could be a legal or data-integrity/provenance related reason to refer to another external resource. For example, the OGC API Features supplies a local versioned-based database of features referred by another authority entity that does a poor job at maintaining them (e.g.: they simply override or preserve the latest features). The canonical link would point at that original entity, while the current would point to the latest version maintained locally.

An issue would be that, if OGC API Features specifies a specific semantic of canonical in the versioning context, even if it is optional, that way of using it for authoritative reference would not be possible anymore since the client receiving the response with these links would have no way to know which concept it refers to (versioning or authority).

Since the ones from RFC 5289 are explicitly for versions, we should use those for the series of versions and first, last, prev, next for pages. I will update the proposal accordingly.

Agreed.

---

I do not understand this discussion. Most of the proposal discusses how to access particular versions of a feature. See Access to a feature and its versions and the related examples. For example, GET /api/collections/building/items/1?datetime=2005-01-01T00:00:00Z provides access to the version of feature "1" on January 1st, 2005.

In case (2), using /collctions/xx/items?revisions=true could allow returning all revisions of a feature (with additional filters applicable) rather than only the latest one (default behaviour).

This is covered by / the same as /api/collections/building/items?datetime=../..

Maybe it is due to a lack of Fetures-specific (and maybe too much STAC-specific) understanding on my end, but I believe that if you had 2 versions of a feature that overlapped datetime (as in, their data not versioning), that would be somewhat ambiguous. Assuming datetime is for the data, it should not directly impact versioning.

For example, if Feature "1" v1 and v2 had the same datetime, but updated some other details like the geometry or properties, ?datetime= wouldn't be able to distinguish them (and it shouldn't by design). For a client that doesn't know how to interpret versioned features, only v2 would be returned and all works as usual. But for a client that does support versioned features, how does it reach v1 if the same featureId: "1" is used? I think that would be the use case of ../items/1?version=v1 and ../items/?revisions=true. The datetime would remain only to filter the datetime property of the features across versions, but accessing those specific versions requires another parameter to deal with overlaps.

For completion, if datetime does not overlap between versions, that is just a nice convenience that v1 and v2 could be filtered automatically by the right datetime range/instant combined with ?revisions=true. However, omitting ?revisions=true and giving an out-of-range datetime should NOT return v1. From the perspective of a non versioned-aware client, v1 doesn't exist anymore. The ?datetime parameter should only match "latest" versions (v2 in this case). Therefore, out-of-range datetime would yield no match.

To fetch multiple versions of a feature in a single request, the Features (items) resource can be used, with a datetime parameter that is an interval and an ids parameter to select the feature - or a CQL2 filter.

That is part of my worry. If a client wants the revisions or a single older version of a feature, they need to craft a CQL2 filter based on possibly inconsistent definitions of version/datetime/other properties combinations across API implementations, since each implementation could encode the feature ID differently. Worst to that, they need to combine that CQL2-versioning filter to possibility other CQL2 conditions needed to filter other non-versioning considerations.

Discovering what is the right CQL2 to craft seems overly cumbersome, prone to errors, and not really interoperable. On the other hand, dedicated version/revisions parameters would make the intent very explicit and interoperable (non versioned-aware clients just ignore it), without overloading the meaning of datetime between data/versioning concepts.

Also, from the perspective of clients, they do not even see which versions are there originally because the API returns (by design) only the latest feature versions by default. Finding how the fields are encoded to obtain the revisions therefore even harder to discover reliably.

[fmigneault]

Item raised during the OGC API SWGs track - Helsinki by @sampov2 :

features-with-time-slices seems a bit weak since it doesn't cover cases where the geometry changes

I agree with that.