Buffer Pool abstraction

[vbe0201]

io_uring blesses us with provided buffers, a feature where we can preregister pools of buffers and have operations like recv pick an available buffer. This not only avoids data copies between kernel and user space, it gives us a convenient primitive for efficient buffer reclamation and reuse.

User Interface

There are different approaches to designing the user interface. The following lists a couple approaches with their tradeoffs for discussion.

The low-level implementation just requires associating the ID of a pool with a syscall that should pick an available buffer from it, so we have a lot of freedom with how we want to abstract this.

Per-ring pool through RingConfig

This is by far the simplest but also the least flexible approach. You specify the number of buffers in your pool and the size of each individual buffer in bytes through the RingConfig and boros sets up the pool implicitly:

# 128 buffers of 1KB each
conf.pool_size = 128
conf.buf_size = 1024

The pool is then used by default where it is applicable.

Drawbacks:

• No possibility to opt out of using a provided buffer except with explicit read/read_pooled APIs.
• No possibility to have multiple buffer pools. E.g. different subsystems in an application may want different pools to limit their resource consumption.

Explicit pool management

This is the most flexible approach but introduces most burden on the end user - all pools are set up manually and must be passed to an API whenever it should be used.

pool = BufferPool(128, 1024)
buf = await client.read(pool)

Drawbacks:

• May become overly verbose since pooled memory is always a good default
• Can easily be forgotten if pool parameter is optional.

Pinning pools to resources

Something in-between the two:

client_pool = BufferPool(128, 1024)

# receiving data from the client will use the pinned pool by default
# if no pool is pinned, memory will be allocated on the fly
client.pin(client_pool)
buf = await client.read()

Drawbacks:

• Arguably even easier to forget.

Soundness concerns

• Buffers must not be destroyed before an operation completes and must stay pinned to their memory locations for the duration of an operation.

  • In CPython, address stability is a given since the id() builtin relies on it. And to keep buffers alive, we can simply bump the refcount of the pool for the duration of an operation.

• A buffer may not be accessed for reading or writing from the Python side while it's in use.

  • With pools, this is not much of an issue since users always specify a whole pool and not individual buffers from it. However, it gets trickier when we want to pass buffers to an API like send. But that is probably for another issue on the general structure of buffers used in boros.