Revise cli tests #365
Revise cli tests #365
Conversation
- change `convert` to `magick` and `compare` to `magick compare` - change order of arguments to put input file before operations - in computation of diff, remove "-background white -flatten" but I do not know why
|
Do we need to check for gregorio version? Or is it good enough to change: to:
|
- change `convert` to `magick` and `compare` to `magick compare` - change order of arguments to put input file before operations - in computation of diff, remove "-background white -flatten" but I do not know why
Compatibility with ImageMagick 7
By splitting the testing of stderr and stdout output, we get more information about what is going wrong when a test fails.
|
Okay, that should rewrite all the cli tests so that the actual output messages are checked and not the exit code. Now that I write that, I realize that I should probably "add" tests that test the exit state. Of course, those would simply be the original tests (or at least equivalent to them). |
While one could argue that the original tests were based really tests of the exit state, I decided not to rewrite the tests in a way that makes explicit that we are testing the exit state and what we expect the exit state to be. Since we only use exit states of 0 (success) and 1 (failure) that may seem like overkill, but this leaves open the possibility of us making a more expansive use of exit states in the future.
The error messages for a few tests use the actual name of the invocation rather than just "gregorio" in a few more places than I originally thought.
|
So, that completes the cli tests, but I've been looking at the other scripted tests and am now wondering if I shouldn't do the same thing for them. Any thoughts on this? |
|
Can anyone explain the difference between the two kpsewhich tests? |
Still need to do the kpsewhich tests, but I need to understand the difference between them first.
|
Okay, I've figured out what I've double checked that I'm using the gregorio that comes with TeX Live (i.e. one which is compiled with the kpathsea libraries). This may be revealing a problem with our use of the kpathsea libraries. Can anyone confirm that? |
Here's a sample of how I'm thinking of revising the scripted cli tests. The basic idea is to capture the expected output (either to stdout or stderr or both) and make sure it's correct. Right now I've not expanded the number of tests, but I could see testing the stdout output and the stderr output in separate tests (e.g. stdin_and_stdin.sh would become 2 tests).
Some form of VersionManager.py (from the main repository) will probably need to be added to the repository for this to work (as version number appears in some output, e.g. version.sh).
What do people think about this revision?