test: run the conformance corpus from Tish, the third target - #12
Merged
Conversation
The corpus exists to make "one source, three targets" a fact rather than a claim, and it was only ever run by two of them: the JS build and the Rust crate. The Tish source itself — what both are generated FROM — was never checked against it. A change that altered the Tish semantics but survived both emits would have gone unnoticed by the very thing meant to catch it. test/conformance.tish imports ../src/index.tish and runs the same inputs against the same expected files. All 11 cases pass, so the three targets are now demonstrably in agreement rather than assumed to be. Two things it has to do differently, both noted where they happen: Run under `tish run` instead of the JS build, because reading the corpus needs `tish:fs` and the JS target has no filesystem at all. Compare semantically rather than textually. This runtime's JSON.stringify ignores its indent argument and emits compact output, so comparing against the pretty-printed expected file failed on whitespace for all 11 cases while the data matched exactly. Re-stringifying the parsed expectation puts both sides in one form; key order survives the round-trip, so real drift is still caught.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The corpus exists to make "one source, three targets" a fact rather than a claim. It was only ever run by two of them — the JS build and the Rust crate. The Tish source itself, which both are generated from, was never checked against it.
That's the leg that mattered most: a change altering the Tish semantics but surviving both emits would have gone unnoticed by the very thing meant to catch it.
test/conformance.tishimports../src/index.tishand runs the same inputs against the same expected files.Two things it does differently
Runs under
tish run, not the JS build. Reading the corpus needstish:fs, and the JS target has no filesystem —readDirisn't defined there under any feature flag.Compares semantically, not textually. This runtime's
JSON.stringifyignores its indent argument and emits compact output, so comparing against the pretty-printed expected files failed all 11 cases on whitespace while the data matched exactly. Re-stringifying the parsed expectation puts both sides in one form; key order survives the round-trip, so real drift is still caught.Wired into
npm testand CI as its own step.