diff --git a/README.md b/README.md index 8785e57c..a04a99c2 100644 --- a/README.md +++ b/README.md @@ -86,7 +86,7 @@ These are: 1. `optional/`: Contains tests that are considered optional. Note that this subdirectory currently conflates many reasons why a test may be optional -- it may be because tests within a particular file are indeed not required by the specification but still potentially useful to an implementer, or it may be because tests within it only apply to programming languages with particular functionality (in which case they are not truly optional in such a language). In the future this directory structure will be made richer to reflect these differences more clearly. -2. `proposals/`: Contains a subfolder for each active proposal to the specification. If the proposal is a keyword (generally the case), then the subfolder will bear the name of that keyword. Inside the proposal subfolder is a series of test files that would contain amendments to the required test suite should the proposal be incorporated into the specification. These test should be considered volitile while the proposal is in development, however implementations claiming to support the proposal are expected to pass its tests. +2. `proposals/`: Contains a subfolder for each active proposal to the specification. If the proposal is a keyword (generally the case), then the subfolder will bear the name of that keyword. Inside the proposal subfolder is a series of test files that would contain amendments to the required test suite should the proposal be incorporated into the specification. These tests should be considered volatile while the proposal is in development; however, implementations claiming to support the proposal are expected to pass its tests. ## Using the Suite to Test a Validator Implementation diff --git a/annotations/README.md b/annotations/README.md index 69cd3dd7..c13ab262 100644 --- a/annotations/README.md +++ b/annotations/README.md @@ -5,7 +5,7 @@ on which values of an instance. These tests are agnostic of any output format. ## Supported Dialects -Although the annotation terminology of didn't appear in the spec until 2019-09, +Although annotation terminology didn't appear in the spec until 2019-09, the concept is compatible with every version of JSON Schema. Test Cases in this Test Suite are designed to be compatible with as many releases of JSON Schema as possible. They do not include `$schema` or `$id`/`id` keywords so @@ -25,7 +25,7 @@ A short description of what behavior the Test Case is covering. The `compatibility` option allows you to set which dialects the Test Case is compatible with. Test Runners can use this value to filter out Test Cases that -don't apply the to dialect currently under test. The terminology for annotations +don't apply to the dialect currently under test. The terminology for annotations didn't appear in the spec until 2019-09, but the concept is compatible with older releases as well. When setting `compatibility`, test authors should take into account dialects before 2019-09 for implementations that chose to support @@ -42,7 +42,7 @@ Case is compatible with draft-07 and up. **Example**: `"compatibility": "7"` You can use a `<=` operator to indicate that the Test Case is compatible with -releases less then or equal to the given release. This example indicates that +releases less than or equal to the given release. This example indicates that the Test Case is compatible with 2019-09 and under. **Example**: `"compatibility": "<=2019"` @@ -111,6 +111,6 @@ could contribute annotations for a single keyword. An empty object is an assertion that the annotation must not appear at the `location` for the `keyword`. -As a convention for this Test Suite, the `expected` array should be sorted such +As a convention for this Test Suite, the `expected` object should be sorted such that the most recently encountered value for an annotation given top-down evaluation of the schema comes before previously encountered values. diff --git a/annotations/test-case.schema.json b/annotations/test-case.schema.json index 6df5f109..1c1bd337 100644 --- a/annotations/test-case.schema.json +++ b/annotations/test-case.schema.json @@ -13,7 +13,7 @@ "pattern": "^(<=|=)?([123467]|2019|2020)(,(<=|=)?([123467]|2019|2020))*$" }, "schema": { - "markdownDescription": "This schema shouldn't include `$schema` or `id`/`$id` unless necesary for the test because Test Cases should be designed to work with as many releases as possible.", + "markdownDescription": "This schema shouldn't include `$schema` or `id`/`$id` unless necessary for the test because Test Cases should be designed to work with as many releases as possible.", "type": ["boolean", "object"] }, "externalSchemas": { diff --git a/output-tests/README.md b/output-tests/README.md index d209bdb2..d19fbdd2 100644 --- a/output-tests/README.md +++ b/output-tests/README.md @@ -8,7 +8,7 @@ _**NOTE** Although the formats didn't change much between 2019-09 and 2020-12, t The tests are organized by specification release and then into two categories: content and structure. -Content tests verify that the keywords are producing the correct annotations and/or error messages. Since there are no requirements on the content of error messages, there's not much that can be verified for them, but it is possible to identify when a error message _could_ be present. Primarily, these tests need to extensively cover the annotation behaviors of each keyword. The only output format needed for these tests is `basic` for 2019-09/2020-12 and `list` for later versions. +Content tests verify that the keywords are producing the correct annotations and/or error messages. Since there are no requirements on the content of error messages, there's not much that can be verified for them, but it is possible to identify when an error message _could_ be present. Primarily, these tests need to extensively cover the annotation behaviors of each keyword. The only output format needed for these tests is `basic` for 2019-09/2020-12 and `list` for later versions. Structure tests verify that the structures of the various formats (i.e. `flag`, `basic`, `detailed`, `verbose` for 2019-09/2020-12 and `flag`, `list`, `hierarchical` for later versions) are correct. These tests don't need to cover each keyword; rather they need to sufficiently cover the various aspects of building the output structures by using whatever keywords are necessary to do so. @@ -22,7 +22,7 @@ The `output` property itself has a property for each of the output formats where ## Other notes -### Ambiguity around 2020-09/2020-12 `basic` +### Ambiguity around 2019-09/2020-12 `basic` The 2019-09/2020-12 specs don't define the structure of `basic` very thoroughly. Specifically there is a nuance where if the list contains a single output node, there are two possible structures, given the text: diff --git a/tests/v1/proposals/README.md b/tests/v1/proposals/README.md index 4cb65322..2a347583 100644 --- a/tests/v1/proposals/README.md +++ b/tests/v1/proposals/README.md @@ -1,10 +1,10 @@ # JSON Schema Proposals Tests -The subfolders herein represent tests for each active proposal to the specification. If the proposal is a keyword (generally the case), then the subfolder will bear the name of that keyword. Inside the proposal subfolder is a series of test files that would contain amendments to the required test suite should the proposal be incorporated into the specification. These test should be considered volitile while the proposal is in development, however implementations claiming to support the proposal are expected to pass its tests. +The subfolders herein represent tests for each active proposal to the specification. If the proposal is a keyword (generally the case), then the subfolder will bear the name of that keyword. Inside the proposal subfolder is a series of test files that would contain amendments to the required test suite should the proposal be incorporated into the specification. These tests should be considered volatile while the proposal is in development; however, implementations claiming to support the proposal are expected to pass its tests. > [!IMPORTANT] > This folder is NOT intended for proposed additions to the test suite or the specification. Proposals to the test suite can be made by [opening an issue](https://github.com/json-schema-org/JSON-Schema-Test-Suite/issues/new) in this repository. -Proposals to the specification should be made by following the process defined by our [Feature Life Cycle](https://github.com/json-schema-org/json-schema-spec/blob/main/PROCESS.md#feature-life-cycle). \ No newline at end of file +Proposals to the specification should be made by following the process defined by our [Feature Life Cycle](https://github.com/json-schema-org/json-schema-spec/blob/main/PROCESS.md#feature-life-cycle).