chore: Refactor integration tests

[stevehipwell]

This PR replaces #2476.

---

Before the change?

• The acceptance tests are complicated and don't all pass

After the change?

• Acceptance tests have been refactored
  • Fewer tests
  • Simpler test structure
  • Test env setup has been made more granular and documented
• Acceptance tests have been tested
  • Tests introduced after v6.4.0 haven't been tested
  • Enterprise tests and external user tests have been skipped

Outstanding

• Re-run all tests locally

Pull request checklist

• Schema migrations have been created if needed (example)
• Tests for the changes have been added (for bug fixes / features)
• Docs have been reviewed and added / updated if needed (for bug fixes / features)

Does this introduce a breaking change?

Please see our docs on breaking changes to help!

• Yes
• No

---

[nickfloyd]

These changes are fantastic! There was a lot, thank you for putting in the 💪 here! I think the only thing that stood out to me is that the workflows / ci jobs need to be updated now that you've change the environment vars - I saw that you updated the contributing doc.

[stevehipwell]

Thanks @nickfloyd I had meant to add that to the description as work to do, as I removed the actual automation changes for a later PR. I also need to do a clean review (not straight after making the changes) to check that nothing got screwed up in the merge.

[stevehipwell]

@nickfloyd this PR depends on #2946 to be able to configure the automation tests.

[stevehipwell]

@nickfloyd I think these are ready for a final review. They're in better shape than the ones currently on main and it'll be easier to get them working locally and in automation once they're merged.