New features for yml2vocab...

[iherman]

Hi @msporny @pchampin @BigBlueHat

this is just a discussion/information about some features I am playing with on yml2vocab. In the latest (not-yet-merged and published) version:

• Added the import block to the json_ld top level block, to allow adding a @import statement with one or more URI-s at the top of the generated context file.
• When generating a context file with an aliased term (see the known_as property) an extra information is added to the generated HTML file.
• Added the capability to handle HTML templates containing HTML fragments only. The result can be imported into a specification instead of keeping it as a separate file.
• Introduced the upper_union and range_union boolean flags for classes and, respectively, properties. If set to true, the superclass, respectively range settings with several class references mean the disjunction of the classes, as opposed to the (default) conjunction. This formally affects the turtle and json-ld versions of the vocabulary (which use the owl:unionOf construct); the HTML description uses now the ⊓ and ⊔ characters to denote these. This allows to say: "instances of this class are of type either this, this, or this".
• Introduced the one_of keys for classes and datatypes. The values list the possible individuals for a class, and the possible string values for a datatype derived from xsd:string. Furthermore, the pattern key for a datatype can hold a regular expression restricting the acceptable string value.
• Some minor optimization (e.g., removal of unnecessary prefix definition from the generated html/turtle/jsonld).
• Re-edited the README.md file to make it somewhat more readable.

I am still wondering about extending the 'upper_union' to datatypes, allowing new datatype inhering from A or B. It is straightforward now to add it, but I do not whether it is of practical use.

These changes came to the fore because I began to use it on a different project, namely (Benjamin, listen!) an upcoming EPUB Annotation specification. That spec uses a profile of the W3C open annotation spec, meaning using the model but restricting the vocabulary (using only a few annotation types, or selector types, that sort of things).

Any feedback would be welcome!

(This is only discussion, not yet a set of issues... It may come to that.)

[msporny]

Hey @iherman, just some thoughts off of the top of my head on whether or not we'd use these new features:

Added the import block to the json_ld top level block, to allow adding a @import statement with one or more URI-s at the top of the generated context file.

Useful, but I don't think we use @import statements in any of our JSON-LD contexts (because people keep complaining that doing so is a security issue).

When generating a context file with an aliased term (see the known_as property) an extra information is added to the generated HTML file.

What's the extra information added? I expect this to be useful for us.

Added the capability to handle HTML templates containing HTML fragments only. The result can be imported into a specification instead of keeping it as a separate file.

This sounds very useful and would replace some home grown tooling we have to do this same thing.

Introduced the upper_union and range_union boolean flags for classes and, respectively, properties. If set to true, the superclass, respectively range settings with several class references mean the disjunction of the classes, as opposed to the (default) conjunction. This formally affects the turtle and json-ld versions of the vocabulary (which use the owl:unionOf construct); the HTML description uses now the ⊓ and ⊔ characters to denote these. This allows to say: "instances of this class are of type either this, this, or this".

This is probably useful to us, but I am concerned that the ⊓ and ⊔ characters will be difficult to read for some of our communities, such as healthcare, banking, etc. We plan to use the direct output of yml2vocab for these communities, who do have vocabulary maintainers/practioners, but we want the vocabulary documents to be largely readable by people without a lot of background knowledge on ontology design terminology. It would be nice if there was a "broadly human readable" vs. "readable by a trained ontologist" outputs -- I know that might be asking for too much, though.

Introduced the one_of keys for classes and datatypes. The values list the possible individuals for a class, and the possible string values for a datatype derived from xsd:string. Furthermore, the pattern key for a datatype can hold a regular expression restricting the acceptable string value.

Very useful, we will use this feature.

Some minor optimization (e.g., removal of unnecessary prefix definition from the generated html/turtle/jsonld).

That is great and helpful, as we have moved away from using prefixes if at all possible in our JSON-LD Contexts.

Re-edited the README.md file to make it somewhat more readable.

Very useful as well. :)

I am still wondering about extending the 'upper_union' to datatypes, allowing new datatype inhering from A or B. It is straightforward now to add it, but I do not whether it is of practical use.

I don't know if we have a use case for this as we try to keep datatype definitions to a minimum, though I do appreciate the theoretical usefulness and completeness of having this feature.

[iherman]

Hey @iherman, just some thoughts off of the top of my head on whether or not we'd use these new features:

Added the import block to the json_ld top level block, to allow adding a @import statement with one or more URI-s at the top of the generated context file.

Useful, but I don't think we use @import statements in any of our JSON-LD contexts (because people keep complaining that doing so is a security issue).

I know. But EPUB, though security is always important, is less critical in this respect. The requirement for developers to include two context references instead of one may become an issue. It is open for us at this moment.

When generating a context file with an aliased term (see the known_as property) an extra information is added to the generated HTML file.

What's the extra information added? I expect this to be useful for us.

A warning that the term used in JSON-LD is not the same as the official vocabulary name.

Added the capability to handle HTML templates containing HTML fragments only. The result can be imported into a specification instead of keeping it as a separate file.

This sounds very useful and would replace some home grown tooling we have to do this same thing.

This is still in consideration for EPUB, but I added the feature just in case.

Introduced the upper_union and range_union boolean flags for classes and, respectively, properties. If set to true, the superclass, respectively range settings with several class references mean the disjunction of the classes, as opposed to the (default) conjunction. This formally affects the turtle and json-ld versions of the vocabulary (which use the owl:unionOf construct); the HTML description uses now the ⊓ and ⊔ characters to denote these. This allows to say: "instances of this class are of type either this, this, or this".

This is probably useful to us, but I am concerned that the ⊓ and ⊔ characters will be difficult to read for some of our communities, such as healthcare, banking, etc. […] It would be nice if there was a "broadly human readable" vs. "readable by a trained ontologist" outputs -- I know that might be asking for too much, though.

I was concerned by this myself. To be honest, I have not found a proper replacement yet, but I am obviously biased by my mathematical background. An English "prose", like "A or B or C" and "A and B and C" is not necessarily clearer, and there is not much space in the current layout to add a very verbose text. Alternatives (though mathematically less precise) signs like ⋀ and ⋁ or ⋂ and ⋃, which are probably more widely known, are not much better.

Ideas are welcome.

Introduced the one_of keys for classes and datatypes. The values list the possible individuals for a class, and the possible string values for a datatype derived from xsd:string. Furthermore, the pattern key for a datatype can hold a regular expression restricting the acceptable string value.

Very useful, we will use this feature.

Knowing that, in reality, there is not much use for it. It is represented in the vocabulary, but I do not think it is possible to do it in a context file.

Ideally, it would be better to generate a JSON Schema and/or a SHACL. The latter is not possible, because SHACL (even in the latest versions) does not include datasets. I would not even dare to get into the former: I am not an expert in Schemas and I seriously doubt it is 100% doable.

Note that a SHACL would need a serious surgery on the code. At the moment the simplified approach is to generate Turtle and JSON-LD directly from the internal representation of the data generating, in the case of Turtle, simply text. Adding SHACL would probably require using a more complex but somewhat "cleaner" approach, namely to convert the internal representation into RDF using some RDF Runtime and then use the runtime's serializers. Doable, but takes a lot of time.

[iherman]

@llemeurfr you might be interested in this discussion, too

[msporny]

Ideally, it would be better to generate a JSON Schema and/or a SHACL.

Yes, exactly, this is why it's important. My hope is that, in time, we'll be able to auto-generate both JSON-LD Contexts and JSON Schema from the yml2vocab file.

[iherman]

Ideally, it would be better to generate a JSON Schema and/or a SHACL.

Yes, exactly, this is why it's important. My hope is that, in time, we'll be able to auto-generate both JSON-LD Contexts and JSON Schema from the yml2vocab file.

Hope is a beautiful thing!😀 But I do not think I would be able to do that... Also, my fear is that it would bring way too many features into the yml file that makes no sense in RDF, and it will make the tool very difficult to comprehend.

[msporny]

I do not think I would be able to do that...

Ah! I didn't mean to imply that you'd do the work. I'm trying to say that as long as the information is in the yml2vocab file, and is valid, other tooling can be built that uses the yml2vocab file as input to produce a JSON Schema.

[iherman]

Understood 😀 I would welcome any contribution!

[iherman]

Introduced the one_of keys for classes and datatypes. The values list the possible individuals for a class, and the possible string values for a datatype derived from xsd:string. Furthermore, the pattern key for a datatype can hold a regular expression restricting the acceptable string value.

Very useful, we will use this feature.

I have added this feature to property ranges as well, also inspired by a (separate) comment of @pchampin on how to use that in a context. It is now possible to say

property:
    - id: highlight
      domain: Body
      one_of: [solid, underline, strikethrough, outline]
      defined_by: https://w3c.github.io/epub-specs/epub34/annotations/#1-4-body

it will create, in the ontology, an anonymous class that consists of these individuals, and use it as a range. Ie, it restricts the values of the property. (solid, underline, etc, are to be defined as individuals in the vocabulary) And here comes the trick of @pchampin : the tool will put into the context file something like:

        "highlight": {
          "@id": "https://www.w3.org/ns/epub/annotation/vocabulary#highlight",
          "@type": "@vocab",
          "@context": {
            "solid": "https://www.w3.org/ns/epub/annotation/vocabulary#solid",
            "underline": "https://www.w3.org/ns/epub/annotation/vocabulary#underline",
            "strikethrough": "https://www.w3.org/ns/epub/annotation/vocabulary#strikethrough",
            "outline": "https://www.w3.org/ns/epub/annotation/vocabulary#outline",
            "@vocab": "ea:INVALID_VALUE:"
          }
        },

Neat.

[iherman]

B.t.w., you can see things in actions, ie, the generated files and the core vocab using some of the new features in: https://github.com/iherman/ea-vocabulary

[pchampin]

all these additions look really nice. Makes me really want to play with yml2vocab more :-)
In fact, I need to introduce the LWS WG to this tool, which would give me an excuse to dig into it and possibly contribute.

Regarding upper_union for datatypes: I think there are cases where it makes a lot of sense, e.g. allowing

• xsd:float, xsd:double or xsd:decimal
• xsd:gYear, xsd:gYearMonth, xsd:date, xsd:dateTime, or xsd:dateTimeStamp`
• xsd:double or xsd:integer
  The latter is of particular interest for JSON-LD, because JSON numbers generate literals of either type (42 → "42"^^xsd:integer, 3.14 → "3.14E0"^^xsd:double)