feat: Compile schema.json and bundle in Python package

[effigies]

In bids-standard/python-validator#2, I'm generating classes on the fly from schema.meta.context. As it stands, the import takes about 530ms on my machine, about 470ms of it spent loading the schema and constructing the classes.

This change builds schema.json during the Python packaging process and includes it instead of the YAML directory inside the tarball and wheel. Along with deferring a couple imports (here and in #2109), this brings the import time down to about 35ms, with 10ms spent on loading the schema and building the classes.

Editable installs are left loading YAML files, to avoid caching an old state.

[yarikoptic]

quick one, am I correct to assume that then it would "hardcode" python validator to

• use 1 specific (potentially very old) bids-specification version for that release of bidsschematools?
• we would need to release bidsschematools for every version of bids itself?
• at runtime only 1 that specific version of bids could be validated?

To avoid "yes" to any of those questions, I would have thought for us to keep *tools separate from data. Tools ideally should support multiple versions of the standard overall and ideally even at runtime (e.g. consider validating two different bids datasets of two different versions in the same ipython session). If we are to bundle pip installable copy of standard, we could do it via separate package, may be off https://github.com/bids-standard/bids-schema

edit: nwb&pynwb are experiencing various issues due to pretty much such tight coupling

[yarikoptic]

per my comment in the main thread, I do not think we should ship any version of the schema with the tools.

[effigies]

Wrote a response, but apparently didn't finish or hit "Comment".

The schema is already bundled with the tools. This PR:

1. Allows load_schema() to load a schema from JSON, not just a YAML directory.
2. Replaces a YAML directory with an equivalent JSON during sdist build.
3. Will load either a bundled JSON or YAML directory, to enable regular installs to load fast, while editable installs still get live updates.

If you want to remove the schema from the tools, that will definitely be a breaking change, and will require rethinking the whole package.

[effigies]

@yarikoptic Want to join the schema call to discuss this?

[yarikoptic]

sorry, was/is busy... ok, so it is (as described) just for the sake of speed performance, gotcha... boost is significant indeed, so let's go for it.