revamp testing infra

.pre-commit-config.yaml

@@ -19,7 +19,7 @@ repos:
     hooks:
     - id: flake8
       additional_dependencies: [flake8-bugbear]
-      exclude: ^tests/b.*
+      exclude: ^tests/eval_files/.*
 
   - repo: https://github.com/rstcheck/rstcheck
     rev: v6.2.4

DEVELOPMENT.md

@@ -42,23 +42,48 @@ cd flake8-bugbear
 /path/to/venv/bin/pip install -e '.[dev]'
 ```
 
+## Writing Tests
+
+flake8-bugbear has a test runner that will go through all files in `tests/eval_files/`, run them through the linter, and check that they emit the appropriate error messages.
+
+The expected errors are specified by adding comments on the line where the error is expected. The format consists of the error code, followed by a comma-separated list of the `col_offset` as well as `vars` that are used when `str.format`ing the full error message.
+```python
+x = ++n  # B002: 4
+try:
+    ...
+except* (ValueError,):  # B013: 0, "ValueError", "*"
+    ...
+```
+The error code should be in the `error_codes` dict, and the other values are `eval`'d as if in a `tuple` and should be valid python objects. (I.e. remember to quote strings)
+
+You can also specify options to be passed to `BugBearChecker` with an `# OPTIONS` comments
+```python
+# OPTIONS: extend_immutable_calls=["fastapi.Depends", "fastapi.Query"]
+# OPTIONS: classmethod_decorators=["mylibrary.makeclassmethod", "validator"], select=["B902"]
+```
+
+If you specify a python version somewhere in the file name with `_pyXX`, the file will be skipped on smaller versions. Otherwise the name has no impact on the test, and you can test multiple errors in the same file.
+
+The infrastructure is based on the test runner in https://github.com/python-trio/flake8-async which has some additional features that can be pulled into flake8-bugbear when desired.
+
+
 ## Running Tests
 
 flake8-bugbear uses coverage to run standard unittest tests.
 
 ```console
-/path/to/venv/bin/coverage run tests/test_bugbear.py
+/path/to/venv/bin/coverage run -m pytest tests/test_bugbear.py
 ```
 
 You can also use [tox](https://tox.wiki/en/latest/index.html) to test with multiple different python versions, emulating what the CI does.
 
 ```console
 /path/to/venv/bin/tox
 ```
-will by default run all tests on python versions 3.8 through 3.12. If you only want to test a specific version you can specify the environment with `-e`
+will by default run all tests on python versions 3.9 through 3.13. If you only want to test a specific version you can specify the environment with `-e`
 
 ```console
-/path/to/venv/bin/tox -e py38
+/path/to/venv/bin/tox -e py313
 ```
 
 ## Running linter