Improve make_group/make_acc_group fixture consistency

[asnare]

Changes

This PR implements some fixture improvements that were originally implemented downstream:

• When creating an account or workspace group, we now require the group to be visible via two subsequent .get() and .list() calls. This double-check approach mitigates (but doesn't completely eliminate) problems with consistency of the APIs for working with groups.
• The wait_for_provisioning argument to the make_group and make_acc_group has now been removed: we always wait. (For compatibility the argument is still accepted but will trigger a deprecation warning.)

An incidental change is the internal unit-test plumbing can now be provided with mock fixtures specific to the test; this is needed to ensure the double-check implementation can be tested from the unit tests.

Linked issues

Supersedes databrickslabs/ucx#2634.

Tests

• added/updated unit tests
• existing integration tests

[github-actions]

This PR breaks backwards compatibility for databrickslabs/blueprint downstream. See build logs for more details.

_{Running from downstreams #46}

[github-actions]

✅ 35/35 passed, 3 skipped, 2m30s total

_{Running from acceptance #87}

[asnare]

This PR breaks backwards compatibility for databrickslabs/blueprint downstream. See build logs for more details.

Running from downstreams #32

I don't fully understand why this check is failing…?

[JCZuurmond]

Add some nits, otherwise looks good

[JCZuurmond]

LGTM

[nfx]

don't modify a global variable - this will break unit tests at multi-threaded execution.

[nfx]

this modifies shared global state and will definitely break when running in multiple threads. why is this necessary? revert.

[asnare]

I need this because the default auto-mocks don't allow the double-check waiting inside make_group() to complete: the return values of .create() and .list() on the (account-)groups REST APIs need to be controlled.

At the moment I don't see a test-specific way to pre-configure the mocks that call_stateful() will set up as fixtures for the fixture being tested. (I didn't find any other unit tests that need this, but may have missed it or there's another path to the same thing.)

Given that pre-configuring mocks for something under test is a fairly common thing to do, I came up with this. It's specifically intended to not only be thread-safe but also async-safe: I don't understand why it will break when running in multiple threads?

The main alternatives that spring to mind are:

• Use a specific keyword argument to call_stateful() for this purpose that isn't passed through to the fixture under test.
• Modify the global fixture (inside CallContext) that all users of call_stateful() have available so that it includes the pre-configured mocks that these group-creation tests use. This would be in the same way as make_random (for example) is handled.

Do you have a preference, or is there something else I've overlooked?

[nfx]

do it without changing a global variable - modify call_context right above the result = ctx[some.__name__](**kwargs) (the first alternative)

def apply(ctx: CallContext):
    mock_group = Group(id="an_id")
    ctx['ws].groups.create.return_value = mock_group
    ctx['ws].groups.list.return_value = [mock_group]

ctx, group = call_stateful(make_group, call_context_callback=apply)

[nfx]

do it without changing a global variable - modify call_context right above the result = ctx[some.__name__](**kwargs) (the first alternative)

def apply(ctx: CallContext):
    mock_group = Group(id="an_id")
    ctx['ws].groups.create.return_value = mock_group
    ctx['ws].groups.list.return_value = [mock_group]

ctx, group = call_stateful(make_group, call_context_callback=apply)

[nfx]

lgtm