Using JSON Pointer fragments with any valid JSON or YAML document

[handrews]

I'm spinning this issue off from #71 (comment).

There is a recurring problem with JSON Schema, OpenAPI, and particularly AsyncAPI, where representations with a variety of media types, and sometimes without any formal media type (therefore defaulting to application/octet-stream), need to be connected using URIs (or IRIs).

For the purpose of this discussion, assume that any URI mentioned is a full URI, either because it was written as such, or because it was a relative URI-reference that has already been successfully resolved against the appropriate base URI. All of this applies to IRIs as well.

@ioggstream has asserted (and please correct me if I get this wrong) that if:

1. a URI is present in a document conforming to a specification X (for example, an API description conforming to the OpenAPI Specification)
2. the non-fragment part of the URI can be resolved to a representation
3. the specification X mandates interpreting the representation as a media type (e.g. application/openapi+json or application/openapi+yaml) that supports JSON Pointer fragments
4. the mandated interpretation is successful (e.g. it was parsed into application/openapi+yaml without error)

then:

1. the original media type (such as application/json, application/yaml, or application/octet-stream), if any (e.g. resolving a file: URI might not come with metadata such as a media type) can be ignored in favor of the re-interpreted media type
2. the URI's fragment can be interpreted according to the fragment syntax and semantics of the re-interpreted media type, e.g. JSON Pointer

Given this, I have the following queetions:

1. Does any standard directly support this process and usage?
2. If not, does any standard (such as RFC 3986 §3.5) forbid it?
3. If not, can/should we create a standard that explicitly endorses such a process?

Having this process ratified by a standard would allow doing something like this:

• define application/pointable+json and application/pointable+yaml which are simply application/json and application/yaml with JSON Pointer fragment support
• having specifications such as OpenAPI and even more notably AsyncAPI, which has to incorporate a wide range of schema formats, not all of which have any defined media type, much less one with fragment support, mandate re-interpreting linked resources that can be parsed as JSON or YAML into these media types so that URIs can use JSON Pointer fragments regardless of the media type associated with the representation by the source of that representation

This would solve a great many problems.

For example, in JSON Hyper-Schema, there is an "anchor" keyword for setting the context URI of a link. Often the context URI only needs to change within the instance, which would normally be done with a same-document reference (typically a fragment-only URI reference).

However, JSON Hyper-Schema has to accommodate application/json instances, which cannot use fragments at all. Therefore I added the "anchorPointer" keyword to separately store a plain-text JSON Pointer to adjust the context separately from the URI in "anchor".

If JSON Hyper-Schema could mandate that, for the purpose of the "anchor" keyword, the instance MUST be interpreted as application/pointable+json, then we could dispense with "anchorPointer" entirely.

While work on JSON Hyper-Schema is tabled and may never resume, this pattern has appeared elsewhere. I'm just too lazy at the moment to figure out a realistic AsyncAPI example.

[handrews]

BTW if there's interest in an IETF-format draft proposal for application/pointable+json and application/pointable+yaml, I would be happy to write that up, or make a PR to add it to the existing API media types spec.

[darrelmiller]

If we consider a URL in a document some form of link, then I think it is feasible to consider that the host document may have conferred some kind of link type based on the location of the URL in the document.
If we consider the wording from RFC8288,

Link relation types can also be used to indicate that the target
resource has particular attributes, or exhibits particular
behaviours;
https://www.rfc-editor.org/rfc/rfc8288#section-2.1

I don't think it is unreasonable to claim that the semantics of the link type defined by the host document would allow using the fragment identifier to reference into the returned representation.

It would be interesting to hunt around to see if there are existing link relations that effectively "re-interpret the media type". Could we argue that the "privacy-policy" link relation "reinterprets" a text/plain representation to have the semantics of a "privacy-policy" document?

Having said this RFC8288 goes on to say,

Relation types are not to be confused with media types [RFC2046];
they do not identify the format of the representation that results
when the link is dereferenced.

Is it valid to make a distinction between a link type "identifying the format" and "allowing a client to re-interpret the media-type"? Or is this just playing word games?

I think the warning in RFC8288 is intended to prevent creating coupling between the context and the target by predefining what media-type must be returned. That's not what we are trying to do here.

[handrews]

@darrelmiller

Is it valid to make a distinction between a link type "identifying the format" and "allowing a client to re-interpret the media-type"? Or is this just playing word games?

Good question. One thing about fragment best practices is that representations suitable for a particular use ought to support the same fragment syntax/semantics. This is actually part of why JSON Schema defines plain name fragments - they are portable to other plausible representations in ways that JSON Pointers are not. On the other hand, anything that works exclusively with a JSON data model, such as the OAS, can use JSON Pointer fragments consistently.

I wonder if it is valid to say "this resource, regardless of its exact representation, is expected to behave sensibly with this sort of fragment".

But it's odd that RFC 6901 seems to go out of its way to prevent anyone from using JSON Pointer fragments even when they can provably be used sensibly:

That is, just because a document is JSON does not imply
that JSON Pointer can be used as its fragment identifier syntax. In
particular, the fragment identifier syntax for application/json is
not JSON Pointer.