Fallback resource adaptor

[madsbk]

New resource adaptor that uses an alternate upstream resource when the primary throws a specified exception type.

The motivation here is to provide NO-OOM by using managed memory when the primary device resource runs out of memory.

Checklist

• I am familiar with the Contributing Guidelines.
• Python tests
• C++ tests
• The documentation is up to date with these changes.

[wence-]

Let's pre-empt #1661

[wence-]

issue: We are moving away from having templated adaptors and instead just using type-erased resource_ref objects (see #1661). To avoid introducing a new such class here, can we immediately move to the new model:

template<typename ExceptionType = rmm::out_of_memory>
class failure_alternate_resource_adaptor final : public device_memory_resource {
   ...
   failure_alternate_resource_adaptor(device_async_resource_ref primary, device_async_resource_ref alternate) : ... {}
   

Or, if we can't type-erase yet, we should at least accept resource refs as an alternate constructor, and store resource_refs rather than templated type-specific usptreams.

[madsbk]

Done in cbd7f43, which rely on implicit type conversion from device_memory_resource* to device_async_resource_ref

[harrism]

Devil's advocating: couldn't we implement this using failure_callback_resource_adaptor? The callback just needs to know about the alternate MR.

[madsbk]

Devil's advocating: couldn't we implement this using failure_callback_resource_adaptor? The callback just needs to know about the alternate MR.

Not as-is. The callback function in failure_callback_resource_adaptor returns a boolean thus there is no direct way for the callback to return the alternate allocation back to failure_callback_resource_adaptor . Another issue is how to handle the deallocation, an allocation allocated by the alternate resource cannot be deallocated by the primary resource.

It is possible to write a new resource that can handle both failure_callback_resource_adaptor and failure_alternate_resource_adaptor but I think this is a case were two simple resources are better than a complex one.

[harrism]

Mostly doc comments. However also needs C++ tests.

[harrism]

I think it's ExceptionType. But actually I don't think you can say what type of exception will be thrown by alternate_upstream if it fails to allocate. It could be a CUDA error, or it could be rmm::out_of_memory, or some other exception.

The alternate upstream should document what exceptions it may throw.

[madsbk]

fixed

[madsbk]

Mostly doc comments. However also needs C++ tests.

Added C++ tests

[harrism]

Looks great, good C++ tests.

[jrhemstad]

nit: fallback_resource_adapater feels like a more concise name.

[harrism]

+1

[madsbk]

+1, will rename.

[madsbk]

done

[wence-]

Approving with request to rename test files appropriately.

[bdice]

We're still discussing the use cases of this feature offline -- for now, I'm requesting changes so that we don't merge this with typos in the API and filename name.

[madsbk]

Let's put this on hold until we get some more use cases.