python: Test against build package

[mpkorstanje]

We should also change our CI so that we test against the built package instead (as mentioned yesterday when fixing the release action)

Originally posted by @youtux in #316 (comment)

Ideally this test done in test-python.yaml.