Add simple start of a test framework

[tangledhelix]

This PR adds the beginnings of an automated test framework for ppgen. This is by no means a complete set of useful tests; in fact it's not looking at the ppgen code at all. There are no unit tests, static analysis, linting, etc. here. This is a purely functional test examining output artifacts, for now.

But we can use the same automation framework to add all the other stuff.

For now, this is what I have:

• A set of ppgen source files (from my own projects)
• Pre-built UTF8 text and HTML outputs from those sources
• Pre-built Guiguts1 .bin and Guiguts2 .json files
• An automation to rebuild the projects with the currently-committed ppgen HEAD
• Automated diff between the pre-built artifact and the newly-built artifact

It's a simple test: if the newly built artifact differs from the pre-built artifact, then the test fails. The idea is to detect unintended differences in output. As the code is changed, some changes in output will be expected, in which case the expected output should be updated so that the test will again pass.

The test projects I've included are not designed to cover all of ppgen's features, so other projects should be included (or invented) to cover more of ppgen's features.

Additionally, it makes sense to do some other things that are more akin to the "usual" sort of automated testing projects might do. I'll point to some of the things we've done in Guiguts 2 as inspiration:

https://github.com/DistributedProofreaders/guiguts-py/blob/master/.github/workflows/python-app.yml

• Linting (we used flake8 and pylint)
• Automated code formatting (we used black; I think uv can do it as well)
• Static type checking (we used mypy)
• Custom unit tests (e.g. pytest)

[tangledhelix]

There are more possible artifact-based tests that could be done here, if desired, too. For example, I didn't include images to test -img, I didn't build Latin-1 files, didn't test -sbin, didn't use a an ini file, etc.

[tangledhelix]

How to test this:

• Checkout my branch (or make your own branch off of it)
• In the "actions" tab in Github, it should run OK (all green) after push
• Make some change either in tests/*/expected/* files, or in ppgen.py that would modify output behavior
• commit & push those changes
• check Actions tab; if the output changed from what was expected, it should fail

At least, if your change would create a diff. Reminder, my test projects don't exercise every ppgen feature, so not every change of output behavior is currently going to be caught by this test framework. It's just a start.

[windymilla]

@tangledhelix - I find it very convenient to be able to run the GG2 tests locally, though some may prefer to use Github actions. GG2 can use pytest to run tests like the ones you are describing, loading an input file, doing something, then checking the output against the expected.
Probably not immediately, but may be worth considering for the future?

ETA: Should have made clear, I don't use ppgen, so this isn't a request for a feature for me - just mentioning what I find helpful in the GG2 setup.

[wf49670]

@tangledhelix -- Tnhanks, Dan. I'll take a look at that. I do something (or have done, in the past, not recently) similar on a more
manual basis. I have a library of ppgen source files, and the output files (html, txt, .bin, etc.) produced by each file. And for new changes I have a .bat file that runs all of the source through the new ppgen version, then does a winmerge "diff" of all the output files which I then examine for any differences. After I accept all the differences as expected, or change the code to eliminate them, I save the new set of output files to compare against the next time.

So, my tests run locally, as @windymilla described for his gg2 testing, but the concept is the same. And I can see that having something less manual, and more centralized where other developers could run it, could be useful.

[tangledhelix]

@tangledhelix - I find it very convenient to be able to run the GG2 tests locally, though some may prefer to use Github actions.

I do the same (for GG2) using a script I made from the Github Actions file. But the point of Actions are that they happen consistently, not just when the developer thinks to do it, and they require no setup. In the case of ppgen, there's no real setup (it only needs Python), but a possible next step in this direction might be to use the matrix functionality to run the same set of tests in all of Python 3.10 through 3.14 (which a developer is not likely to do consistently on their own).

So may aim here is to make something

1. automated
2. consistent
3. requiring no setup on the part of a contributor

But there's nothing stopping anyone from running all the same commands locally, if they like.

[tangledhelix]

For those not familiar with the matrix strategy, a simple example is here. So the idea would be to define whatever tests one wants to run, and then tell the Github machinery to loop through the same thing in various different Python versions. Doing so manually would be tedious.