How to design opendal c binding's async api

[asukaminato0721]

c users have some choices. The main problem is c doesn't have async/await/yield keyword.

• use callback with libuv, libevent
• Manual State Machines
• Macro-based "Coroutines" / Protothreads (something like https://dunkels.com/adam/pt/ or modern version https://github.com/naasking/async.h)
• Coroutine Libraries (https://github.com/semistrict/libcoro or https://github.com/hnes/libaco)

all seems complicated...

the async io part seems also complicated...

---

Refer to #6072 for more context.

[asukaminato0721]

2 ideas:

1. Callback-based API (very common, but call back hell)
2. Event Loop Integration (integrate into libuv, libevent or other framework)

[yuchanns]

Cool. I'm looking forward to it. Go binding can be beneficial from this.

[asukaminato0721]

I made a "callback" version in the PR with test and example, feed back welcome.

[Xuanwo]

Some other things to consider:

• How should we handle the existing tokio code?
• Should we consider splitting the library into a blocking version and an async version?

I can imagine that users who want to use the async API don't want to have tokio included.

---

Or maybe the biggest question is: Where are our C users, and what do they want?

[kpole]

Hi, xuanwo, oceanbase currently uses the blocking version of cbinding and wants to implement async version to reduce the overhead introduced by threading.
In response to your question, I have the following thoughts：

1. As far as I know, tokio has very good asynchronous IO capabilities, and I prefer tokio over writing a C asynchronous framework from scratch
2. Does "splitting the library into a blocking version and an async version" refer to creating two independent libraries, or does it mean two separate sets of APIs?
   • The former might lead to redundant inclusion, especially since OceanBase is introduced in the form of RPM packages, which include the Rust standard library implementation.
   • I believe the latter is feasible, as async IO and sync IO have many differences in interface parameters and significant implementation variations.

[Xuanwo]

For 1, so you prefer to use tokio directly instead of exposing callbacks and relying on the async runtime from the C side?

For 2, yes, having two sets of APIs makes sense to me.

[kpole]

yes, this will cut down the expenses of using the cbinding async framework, at least for what we're doing.

[Xuanwo]

Here lies the issue: the existing blocking C API already uses tokio under the hood. Could you share some insights into your design for the async C API?

[kpole]

That's a good question, the existing blocking C API use tokio, but block_on doesn't use tokio's runtime, instand, it operates on the current thread. In contrast, async IO execute at runtime and communicates with external through callbacks or event loops.

I initially thought sync io would be built on top of async io. However, for now, sync io seems more stable, and I prefer to provide a separate api to implement async io for produciton use.

[asukaminato0721]

I've implemented a basic design #6072 for comments.

---

Async APIs

OpenDAL’s C binding mirrors the Rust async operator, but keeps all runtime management on the Rust side so C callers never need to embed Tokio. The design is intentionally future/await centric:

• opendal_async_operator_new builds an async operator that internally holds a clone of the core Operator plus a handle to OpenDAL’s shared Tokio runtime.
• Each async method (*_stat, *_write, *_read, *_delete) immediately returns an opaque opendal_future_* handle. Creating the future is non-blocking—the runtime schedules the real work on its thread pool.
• You stay in control of when to pull the result. Call opendal_future_*_await to block the current thread until the operation finishes, or opendal_future_*_poll to integrate with your own event loop without blocking.
• If you abandon an operation, call opendal_future_*_free to cancel it. This aborts the underlying task and drops any pending output safely.

Because futures carry ownership of the eventual metadata/error objects, the *_await helpers always transfer heap allocations using the same conventions as the blocking API (free metadata with opendal_metadata_free, free errors with opendal_error_free, etc.).

Need non-blocking integration with your own loop? Call opendal_future_stat_poll(fut.future, &out) inside your loop. It returns OPENDAL_FUTURE_PENDING until the result is ready; once it reports OPENDAL_FUTURE_READY, call opendal_future_stat_await exactly once to consume the output.

See examples/async_stat.c for a narrated walkthrough and tests/async_stat_test.cpp for GoogleTest-based assertions that cover both success and error paths.