Support YAML 1.1

[chronus7]

It seems check-jsonschema currently does not support any setting to explicitly parse input as YAML 1.1. I understand that the default is YAML 1.2.x, but YAML also supports directives to specifically define the file's YAML version¹. This directive seems not to be respected by check-jsonschema.

I looked a bit into it: ruamel.yaml generally supports this directive and parses the file accordingly. But this highly depends on the options with which it is invoked. If I got it right, check-jsonschema invokes ruamel.yaml.YAML only with typ="safe" and pure=False or pure=True. pure=True would actually be helpful, as can be seen in the table below, but it is not necessarily used, as it is only selected as a fallback. For it to be applied, parsers.yaml.impl2loader.load requires the first implementation (pure=False) to throw an exception when loading the file, but a file might still be valid YAML 1.2 even though YAML 1.1 was targeted. In that case, the returned data will correspond to YAML 1.2 parsing rules which will not match the users expectations.

As a concrete example, consider the following:

• A simple schema, expecting a single boolean value:

  {"type": "boolean"}

• A YAML file without %YAML directive intended to be interpreted as YAML 1.1, unmarked.yaml:

  on

• A YAML file with %YAML 1.1 directive intended to be interpreted as YAML 1.1, marked.yaml:

  %YAML 1.1
  ---
  on

I tested these files with the various options, that ruamel.yaml allows to be passed. The results can be seen in the following table, where each column denotes the typ and pure parameters applied. invalid means, that the value has been parsed as a string and as such does not adhere to the schema. valid means, that the value has (correctly) been parsed as boolean and thus matches the schema.

file	rt, False	rt, True	safe, False	safe, True	unsafe, False	unsafe, True	base, False	base, True
unmarked.yaml	invalid	invalid	invalid	invalid	invalid	invalid	invalid	invalid
marked.yaml	valid	valid	invalid	valid	invalid	valid	invalid	valid

I am not confident enough to say which of these options is the best to use (or why the current ones are selected), but considering the involved library actually supports this use case, I would be grateful, if it could also be supported by check-jsonschema.

Footnotes

1. https://yaml.org/spec/1.1/#id895631, https://yaml.org/spec/1.2.2/#681-yaml-directives ↩

[sirosen]

Hi there, thanks for coming here with such a nicely researched and well organized issue!

I need to think about what to do here, since I think there are trade-offs no matter what we choose. The underlying libyaml parser is used if pure=False, and if my memory serves that's preferred because it's faster. But, as you saw, it can fail and we retry with the pure Python implementation, which handles some data more accurately.

For most other usage, I suggest avoiding some of the ambiguous "Norway Problem" YAML features, and using the JSON-compatible true and false. But by specifying the version, you've actually made the input unambiguous... So this is new to me, and different.

I don't want to add a bunch of CLI flags or environment variables to control this behavior. There are already too many flags and it's confusing for users. I'll do some reading to see if we can consistently peek at the first line(s?) of the data, or if there's some other way to detect this in order to choose the best parsing mode.

[chronus7]

Hi, I was curious and dug a bit deeper.

But first of all, I totally understand not wanting to add more CLI flags.
Initially I was thinking about adding another filetype, like yaml1.1 as another option to yaml), because in my original issue I have to be able to parse an unmarked file with no as boolean. But as I then noticed (see the table in the original post already), ruamel.yaml does not seem to be able to parse those anyway. PyYAML does, though.

I have two other options in mind:

1. Realise the use of a different typ as another transforms.Transform, as that one allows to modify the instance via modify_yaml_implementation:

   class YAMLRoundTripTransform(Transform):
       def modify_yaml_implementation(self, implementation):
           implementation.__init__(typ="rt", pure=implementation.pure)

   This feels rather unclean and not fitting for transformation at all, though.

2. Abstract the yaml interface and provide another implementation using PyYAML (as optional dependency), that will be automatically selected, if available. Even though this feels horribly implicit, it seems to be the only way to accept YAML 1.1 out of the box (so even without the directive).

These are obviously just general ideas and I do not expect them to be realised as-is, but maybe they give some hints.

I have expanded my test script a bit and also gone for some timing tests, especially since you mentioned the performance implications. typ = 'pyyaml' is a fake value here to select PyYAML as a YAML loader.

Scenario	A	B	C	D	E	F	G
File Content — typ, pure	on	%YAML 1.1 --- on	%YAML 1.1 --- true	on	%YAML 1.2 --- on	true	%YAML 1.2 --- true
pyyaml, False	valid	valid	valid	valid	valid	valid	valid
pyyaml, True	valid	valid	valid	valid	valid	valid	valid
rt, False	invalid	valid	valid	invalid	invalid	valid	valid
rt, True	invalid	valid	valid	invalid	invalid	valid	valid
safe, False	invalid	invalid	valid	invalid	invalid	valid	valid
safe, True	invalid	valid	valid	invalid	invalid	valid	valid
unsafe, False	invalid	invalid	valid	invalid	invalid	valid	valid
unsafe, True	invalid	valid	valid	invalid	invalid	valid	valid
base, False	invalid	invalid	valid	invalid	invalid	valid	valid
base, True	invalid	valid	valid	invalid	invalid	valid	valid

And for timing, using 7 repetitions with 10000 iterations each:

Scenario	A	B	C	D	E	F	G
File Content — typ, pure	on	%YAML 1.1 --- on	%YAML 1.1 --- true	on	%YAML 1.2 --- on	true	%YAML 1.2 --- true
pyyaml, False	5.758s	5.826s	5.74s	5.799s	5.699s	5.717s	5.662s
pyyaml, True	6.526s	6.882s	6.835s	6.596s	6.506s	6.284s	6.487s
rt, False	8.195s	8.764s	8.697s	8.26s	8.661s	8.096s	8.604s
rt, True	8.213s	8.763s	8.677s	8.244s	8.674s	8.12s	8.58s
safe, False	7.545s	7.587s	7.517s	7.604s	7.564s	7.459s	7.521s
safe, True	8.149s	8.58s	8.527s	8.212s	8.406s	8.034s	8.489s
unsafe, False	7.563s	7.557s	7.523s	7.609s	7.533s	7.506s	7.624s
unsafe, True	8.173s	8.55s	8.663s	8.266s	8.574s	8.169s	8.589s
base, False	7.487s	7.5s	7.569s	7.504s	7.481s	7.444s	7.516s
base, True	8.122s	8.512s	8.484s	8.113s	8.471s	8.095s	8.487s

Maybe this can help you with a decision. Should you have a plan and need someone to implement it, let me know. I am willing to put some time into it.

---

For posterity/transparency:

• check-jsonschema: 4.24.0
  • ruamel.yaml: 0.18.14
  • ruamel.yaml.clib: 2.5.0
• PyYAML: 6.0.2

And the validation method, that is used to check the parsing and that is also being timed:

def validate(file: str, typ: str, pure: bool, schema) -> bool:
    ps = check_jsonschema.parsers.ParserSet()
    if typ == "pyyaml":
        load = lambda x: yaml.load(x, Loader=(yaml.CLoader, yaml.Loader)[pure])
    else:
        load = check_jsonschema.parsers.yaml.construct_yaml_implementation(typ=typ, pure=pure).load
    ps._by_tag["yaml"] = load
    data = ps.parse_file(file, "YAML")
    try:
        jsonschema.validate(data, schema)
        return True
    except jsonschema.exceptions.ValidationError:
        return False