Reduce CI run time

[bdraco]

We have quite a few long running tests that take up a few seconds. Even if we only clean up the top 10 we should be able to shave at least a minute off the CI

[webknjaz]

Is this about what pytest --durations shows?

[bdraco]

Yes

[Dreamsorcerer]

Autobahn tests take something like 25% of the CI time, and the tests don't even work...

[bdraco]

#4247 😢