Skip to content

Commit c902da2

Browse files
committed
Notes on adding a test in CONTRIBUTING
1 parent 79d5822 commit c902da2

File tree

1 file changed

+37
-1
lines changed

1 file changed

+37
-1
lines changed

CONTRIBUTING.md

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,45 @@ We heartily welcome pull requests. To ensure an effective contribution, please a
1919
1. For significant, invasive, or output-affecting changes, consider opening an issue for discussion before committing substantial time and effort.
2020
2. If you're new to the project, starting with the [Good first issues](https://github.com/ocaml-ppx/ocamlformat/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+label%3A%22Good-first-issue+%3Agreen_heart%3A%22) can be beneficial.
2121
3. Fork the repository and create your branch from `main`.
22-
4. If you've added code that should be tested, supplement it with tests located in the `test/` directory.
22+
4. If you've added code that should be tested, supplement it with tests located in the `test/` directory (see [Adding a test](#adding-a-test)).
2323
5. Ensure that the test suite passes (see [Running the tests](#running-the-tests) for instructions).
2424

25+
### Adding a Test
26+
27+
#### Unit Tests
28+
29+
Unit tests are located in the `test/unit` directory and are written using [Alcotest](https://github.com/mirage/alcotest).
30+
31+
#### cram-Style Tests
32+
33+
The CLI tests are located in `test/cli` and are written using [cram testing](https://dune.readthedocs.io/en/latest/reference/cram.html).
34+
Each test is a `.t` file that contains shell commands and their expected outputs.
35+
To add a new test, create a new `.t` file, or add new commands (lines beginning with ` $`) to an existing file.
36+
37+
#### Golden Tests
38+
39+
The remainder of the tests are so called "golden" or "expect" tests. These tests all consist of running `ocamlformat` on
40+
`.ml` files and comparing the output to expected output stored in reference files. The `dune`
41+
configuration for this is automatically generated for you based on the files present in these directories.
42+
43+
These are separated into `test/failing` and `test/passing` directories.
44+
45+
To add a test showing currently incorrect behavior, add a `.ml` file to `test/failing/tests`.
46+
If command line arguments are needed, create a corresponding `.opts` file with the same base name.
47+
The output of these tests will be stored in `.broken-ref` files.
48+
49+
To add a test showing correct behavior, add a `.ml` file to `test/passing/tests`.
50+
These are similar to the failing tests, including the use of the `.opts` files for command line arguments,
51+
however the same file will be tested against all of the built-in configurations (e.g. `default`, `janestreet`, etc.),
52+
with outputs appearing in sub folders named `refs.<configuration>`.
53+
54+
In both cases, if multiple sets of options are desired, you can create
55+
multiple `.opts` files with names provided after a `-`, for example `mytest.ml` can
56+
have `mytest-foo.opts` and `mytest-bar.opts`. This will create multiple test targets.
57+
58+
The first time you run the tests, the `dune.inc` file will be updated. Once you have promoted
59+
this, the tests will run (you may need to manually create blank files for the reference outputs at first).
60+
2561
### Running the Tests
2662

2763
After building OCamlFormat, execute `dune runtest` to check for any regressions. There should be no unexpected diffs. If there are, promote all the diffs with `dune promote`.

0 commit comments

Comments
 (0)