Special handling for non-escaping task-temporary allocations

[manopapad]

Give applications the ability to indicate that a task's temporary pool cannot escape the lifetime of the task (e.g. no output regions and all futures for the task are mapped in advance). Then Legion can pre-allocate the pool and immediately issue a deferred deletion of the backing instance for the pool to Realm such that the deferred deletion can be used to satisfy future deferred allocations.

One thing to confirm, is doing something like this acceptable as a "non-escaping" use?

auto return_buffer = Legion::UntypedDeferredValue(...);
return_buffer.finalize(task_ctx);

Asking because this is what Legate does to e.g. pack multiple return values together.

[lightsighter]

One thing to confirm, is doing something like this acceptable as a "non-escaping" use?

That would be considered an "escaping" use of the pool because the memory we carve out there escapes and is used to store the return value for the future that outlives the task and is likely to be consumed by downstream tasks.

This (non-)escaping property can be set on each memory pool individually though so you can set non-escaping on frame-buffer memory and escaping on zero-copy memory and that way you're only escaping data from "less sensitive" memories if that is an option for you.

[manopapad]

Is there / could there be a way for us to pre-size the output instance during task launch / mapping, and have Legion allocate that for us? Then in that case we can tell you exactly what escapes (only the return value of the task).

[lightsighter]

Is there / could there be a way for us to pre-size the output instance during task launch / mapping, and have Legion allocate that for us? Then in that case we can tell you exactly what escapes (only the return value of the task).

There is not at the moment a way to do this. I think I would be open to doing this if you provided a future size when you register the task variant or launch the task and then we can make that buffer for you in advance and then whatever the first DeferredValue object you make needs to match its size.

[manopapad]

Do you prefer that this case be handled as part of a broader separation between escaping and non-escaping task-temporary allocations? Or OK to have the return value as a special case?

In any case, as you said, it doesn't matter so much in the immediate term, as we can just place those return instances on zcmem.

[lightsighter]

Do you prefer that this case be handled as part of a broader separation between escaping and non-escaping task-temporary allocations? Or OK to have the return value as a special case?

I personally would like to allow you to reserve pools that are both "escpaing" and "non-escaping" and then when you make a DeferredBuffer or a DeferredValue you would have to specify whether you are allocating from the escaping or the non-escaping pool. I remember that @magnatelee thought this would be too hard to do though due to the (batshit crazy) details of the RMM interface which libraries like Legate dataframe have to rely on.

[lightsighter]

To further flush out the semantics: if you allocated a deferred buffer/value from a non-escaping pool and then tried to escape it that would raise an error. If you allocated a deferred buffer/value from an escaping pool and then didn't escape it we would give you a warning.