CI tests stop after first failure

[jpbrodrick89]

I believe this is intentional due to the break statement in tests/__main__.py but worth flagging as an issue for awareness (we ran with pytest for a short period of time so this wasn't an issue then)

# Each file is ran separately to avoid out-of-memorying.
running_out = 0
for file in here.iterdir():
    if file.is_file() and file.name.startswith("test"):
        out = subprocess.run(f"pytest {file}", shell=True).returncode
        running_out = max(running_out, out)
        if out != 0:
            break

However, this means when we accept failure due to bad conditioning which might come up due to random number changes, etc. we might miss more serious failures. Unfortunately, my recent commits have led to some failures on test_well_posed.py (which I will fix, apologies ... yes I do run some tests locally before pushing but due to the long running test suite it's usually just a subset of what might obviously be affected).

Would just removing the break statement be a suitable fix?

[patrick-kidger]

Sounds good to me!

[jpbrodrick89]

Incorporated into #200 hopefully the only failure should be the tests/test_vmap_jvp conlim test (we can add a skip for that if you like, as "test we should fix in the future" or just hardcode a non-random matrix)