handle the case in which the submission ring is full

[carns]

The get_sqe() call in this specific case could fail if no elements are available in the submission ring:

https://github.com/mochi-hpc/mochi-abt-io/blob/main/src/abt-io.c#L2305

In that case, the operation should simply be added back to the head of the working set and break out of the loop. The logic automatically be triggered again the next time a completion event occurs.

There is another get_sqe() call that could fail at this point:

https://github.com/mochi-hpc/mochi-abt-io/blob/main/src/abt-io.c#L2373

In that specific point the mitigation is to simply skip inserting the poll operation. That means that the engine will not be awakened on the next abt-io operation, but that is probably ideal to give the rings a chance to drain before attempting to inject more work.

The other get_sqe() call at https://github.com/mochi-hpc/mochi-abt-io/blob/main/src/abt-io.c#L2268 should just cause a critical failure or maybe trip an assertion. It will be the first operation added to the ring and should never fail if the ring is working correctly.

[carns]

Credit to @chuckcranor for pointing on this case.