Support Schematron validation

[tomschr]

Situation

The README lists some validation with XSD and DTD. However, currently Schematron validation in combination of RELAX NG (RNG) is not supported.

Proposed solution

Schematron rules should be recognized and supported in other schema languages like RNG or XSD.

Relationships

• Related to issue Support RELAX NG validation #450
• Originally posted by @pgundlach in Support RELAX NG validation #450 (comment)

[datho7561]

I am interested in implementing this. If we are not able to support XSD 1.1, this would be a viable way to support some form of assert for XML documents.

It might make sense to include it as an extension to vscode-xml instead of as a part of the base functionality. Not every user will need the Schematron support, and moving the Schematron parts to another extension will reduce the total download for users who don't need Schematron. However, I think its worth seeing the size increase in the LemMinX Jar and binary first before coming to a decision on this.

[angelozerr]

@datho7561 IMHO I think we should start implementing RelaxNG support in #450 before Schematron. In other words, finish and merge the PR eclipse-lemminx/lemminx#841 which provides basic support for RelaxNG (just validation without error range).

This PR uses jing and it seems jing provides support for Schematron too.

[svanteschubert]

RelaxNG is just another grammar aside of DTD and W3C Schema XML and embraced by Sun's Multi Schema Validator.
RelaxNG is the most powerful of the three major grammars (see http://pike.psu.edu/publications/toit05.pdf)
All the above grammars are supported by Sun's Multi Schema Validator, which embraces/supports all of them: https://xmlark.github.io/msv/docs/nativeAPI.html
Therefore I would suggest considering MSV!
I have overtaken Maven release rights from my former colleague Kohsuke Kawaguchi and have to finish a Maven release (https://github.com/xmlark/msv/tree/build-refactoring) and likely repo movement to The Document Foundation (TDF) where LibreOffice is hosted and the ODF Toolkit I am maintaining.
I am the OASIS ODF TC co-chair but on a different leg started yesterday implementing on an opensource syntax binding editor based on your extension for the NGI DAPSI project: https://dapsi.ngi.eu/hall-of-fame/idiss/
Perhaps we can find some synergies over the next months ;-)

[angelozerr]

Therefore I would suggest considering MSV!

Thanks so much @svanteschubert for your comment! We have opened to provide RelaxNG support with any library. Jing seemed is to be the better Java library for RelaxNG, but to be honnest with you I didn't know MSV.

If we wish to consume MSV the first question is what about license? If MSV is not compatible with EPL 2.0 we cannot use it.

Perhaps we can find some synergies over the next months ;-)

It should be fantastic if we could work together to integrate MSV (if license is compatible with EPL 2.0) inside LemMinx. Do you think you could contribute to LemMinx for the RelaxNG support?

[svanteschubert]

@angelozerr The license is BSD.
Oracle has abandoned the project and I have forked officially&legally from the available sources of their last Maven release:
https://search.maven.org/artifact/net.java.dev.msv/msv/2013.6.1/pom
Technically, I was not aware of the Oracle brunch so started to work on https://github.com/kohsuke/msv
That is the reason the copyright header is being added/merged in later. The result will be equivalent in the end!
I believe the fix of copyright headers is the final task before the Maven MSV release (and the repo swift to TDF, RedHead or Eclipse) ;-)

You may see the MSV Maven Oracle history at https://search.maven.org/artifact/net.java.dev.msv/msv

Eclipse mentioned in their FAQ that BSD is allowed to change the license, so I believe we are safe:
https://www.eclipse.org/legal/epl-2.0/faq.php#h.nzy2s8vsuxe2clarify

Regarding your attempt at shanghaiing me - I guess you have success ;-)
I only put my toe into the project this Monday but I would say tentatively yes, but it would help if I could team up later with one of you to have feedback in principle design decision beforehand and not during the pull request review ;-)
Just started and soon summer vacation starts. Would do a new MSV release first to have a stable base and paste here the design suggestions after I have a better understanding of the status quo.

This might be the beginning of a long friendship :)

[angelozerr]

I only put my toe into the project this Monday but I would say tentatively yes, but it would help if I could team up later with one of you to have feedback in principle design decision beforehand and not during the pull request review ;-)

At first we must check MSV can be consumed in LemMinX (it should because BSD and ASF should be compatible with EPL). After that I suggest you provide

• XML validation based on RelaxNG
• XML completion based on RelaxNG

For validation, I suggest you that you see my draft PR to support RelaxNG with Jing eclipse-lemminx/lemminx#841 and adapt it to consume MSV. The hard part in validation is that you should higlight the error as range and not just a given offset. Once validation will be finished, you will need to write test. See https://github.com/eclipse/lemminx/blob/master/org.eclipse.lemminx/src/test/java/org/eclipse/lemminx/extensions/contentmodel/XMLSchemaDiagnosticsTest.java

I suggest you read https://github.com/eclipse/lemminx/blob/master/docs/LemMinX-Extensions.md

If you have another question don't hesitate to ask.

This might be the beginning of a long friendship :)

I hope :)

[Oobiewan]

Hi All,
I am a bit confused about the scope of this issue, is the goal to support schematron for RELAX NG only? Does the extension support schematron for DTD already? I cannot find any mentions of schematron in the description of the extension.
Thanks,
Benedek

[svanteschubert]

All right to be confused. IMHO we talked about Schema, not Schematron. :-)
DTD, XSD, and RNG have all in common to define a relation between parents and their children (attribute, child elements and content - and their datatypes).
Schematron was invented as there are also constraints across the complete full document (let's call them business rules). For example, the invoice net amount of each invoice line of an invoice XML file has to add up to the complete invoice net amount.
Schematron is therefore also used for the EU e-invoice validator:
https://github.com/ConnectingEurope/eInvoicing-EN16931/blob/master/ubl/schematron/UBL/EN16931-UBL-model.sch

The Schematron technology is quite linked to XSLT. These rule files are being transformed into XSLT files doing the validation.
Philip Helger offers some Java-based software for schematron that is also being used in the EU Validator.

Hope I could help
Svante

[datho7561]

I am a bit confused about the scope of this issue,

This issue is for tracking Schematron schema (*.sch) support in lemminx. I believe that you can reference a Schematron file using <?xml-model href="..."?> in any XML document, so that you can validate against a .sch schema, along side other schemas (XSD, DTD, RNG once it's provided).

I started a POC at https://github.com/datho7561/lemminx-schematron based on SchXslt. The issue, as Svante mentioned, is that Schematron is based on XSLT, so we need a library to handle XSLT. My POC is based off of Saxon-HE for the XSLT processing. The issue with this is that Saxon is very large. The main alternative I've look into, Xalan, has some bugs that prevent it from working with SchXslt. I tried building Xalan from sources, but I ran into some issues. Since Schematron is not a priority for us at the moment, I don't have time to work on the POC.

[Oobiewan]

@datho7561 I see, that's a pity, and I'm sorry I don't have the skills to help. Thanks a lot for explaining.
@svanteschubert Thanks for the reply. I see that you did not talk about Schematron, but this issue is about Schematron support according to the title and @datho7561. Now I have the feeling that the issue reporter didn't really know the difference between Schematron and validating against a schema. I just thought it might help everyone involved if we clarify the target here.