Skip to content

Update exchange API to be capsule following new convention #180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
leofang merged 7 commits into from
Jan 24, 2026
Merged

Update exchange API to be capsule following new convention #180

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5eeaaae
Update exchange API to be capsule following new convention
tqchen Dec 5, 2025
1f18a9e
[DOCS] Update the exchange api spec
tqchen Dec 12, 2025
bd96c5f
Update docs/source/python_spec.rst
tqchen Dec 24, 2025
4d25c02
Update docs/source/python_spec.rst
tqchen Dec 24, 2025
68cb293
update per lint
tqchen Jan 9, 2026
29c2602
use commit
tqchen Jan 10, 2026
15127bc
Update to use term thread state
tqchen Jan 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions docs/source/python_spec.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,42 @@ refer to `github.com/dmlc/dlpack <https://github.com/dmlc/dlpack>`_.
guaranteed to be in a certain order or not.


DLPack C Exchange API
~~~~~~~~~~~~~~~~~~~~~

Starting with DLPack 1.3, a new C Exchange API is introduced to enable faster
data exchange than the Python ``__dlpack__`` API at the C extension level.
Producer array frameworks must provide a ``__dlpack_c_exchange_api__``
Comment thread
tqchen marked this conversation as resolved.
Outdated
attribute on the array type.
The attribute should be a ``PyCapsule`` with name ``"dlpack_exchange_api"``.
The consumer can query whether this attribute exists and use it at the C extension level.
Notably, consumer frameworks can always start implementing by only using the Python ``__dlpack__`` API,
and then upgrade to the C Exchange API later when faster data exchange is needed.

.. code-block:: C

PyObject *api_obj = type(tensor_obj).__dlpack_c_exchange_api__; // as C code.
Comment thread
tqchen marked this conversation as resolved.
Outdated
MyDLPackExchangeAPI *api = PyCapsule_GetPointer(api_obj, "dlpack_exchange_api");
if (api == NULL && PyErr_Occurred()) { goto handle_error; }


.. note:: Implementation of the C Exchange API

Producer framework should implement the C Exchange API in a static way either
through Cython, Python C extensions, or Python binding mechanism. Importantly,
because the DLPack C exchange API operates at the C extension level, we need
direct interaction between the array framework PyObject* and DLPack,
Comment thread
tqchen marked this conversation as resolved.
Outdated
as a result it is harder to implement the C Exchange API through ctypes (because
ctypes releases GIL by default and sometimes in non-free-threading environment,
GIL is needed to interact with the Python C API).
Copy link
Copy Markdown
Collaborator

@leofang leofang Dec 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dalcinl or @seberg to comment. I am aware that this is to capture the meeting discussion on why ctypes does not work for our purposes, but I am a bit confused by the mention of GIL.

Copy link
Copy Markdown
Collaborator

@seberg seberg Jan 18, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, this flew past me. Honestly, I am not sure what this note is getting at. The mention of the GIL seems wrong either way. I think I said it in the meeting too, but don't quite remember.

"Releasing the GIL" actually should be thought of "releases the thread-state". And that affects both free-threaded and non-free threaded python just the same.
The difference is that with the GIL a Python thread must own the thread state (and thus the GIL).

I would be fine with just dropping the note, because I am not sure which functions this affects how right now.
The interesting point to make is that some functions are expected to be called without the thread-state (GIL) held, so they need to grab it (like the note that we have for the deallocator).

Copy link
Copy Markdown
Member Author

@tqchen tqchen Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for the note, updated to replace thread state to GIL


A reference implementations of the C Exchange API in frameworks:


* PyTorch: `C++ <https://github.com/pytorch/pytorch/blob/main/torch/csrc/Module.cpp#L692>`__
Comment thread
tqchen marked this conversation as resolved.
Outdated
* Paddle: `C++ <https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/pybind/pybind.cc#L856>`__
Comment thread
tqchen marked this conversation as resolved.
Outdated


Reference Implementations
~~~~~~~~~~~~~~~~~~~~~~~~~

Expand All @@ -198,3 +234,4 @@ ctypes, cffi, etc:
* mpi4py: `Cython <https://github.com/mpi4py/mpi4py/blob/master/src/mpi4py/MPI.src/asdlpack.pxi>`_
* Paddle: `C++ <https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/framework/tensor_util.cc#L901-L951>`__, `Python wrapper using Python C API <https://github.com/PaddlePaddle/Paddle/blob/develop/paddle/fluid/pybind/pybind.cc#L1263-L1280>`__
* Hidet: `ctypes <https://github.com/hidet-org/hidet/blob/main/python/hidet/graph/impl/dlpack.py>`__

23 changes: 12 additions & 11 deletions include/dlpack/dlpack.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
#define DLPACK_MAJOR_VERSION 1

/*! \brief The current minor version of dlpack */
#define DLPACK_MINOR_VERSION 2
#define DLPACK_MINOR_VERSION 3

/*! \brief DLPACK_DLL prefix for windows */
#ifdef _WIN32
Expand Down Expand Up @@ -375,7 +375,7 @@ typedef struct DLManagedTensorVersioned {
} DLManagedTensorVersioned;

//----------------------------------------------------------------------
// DLPack `__c_dlpack_exchange_api__` fast exchange protocol definitions
// DLPack `__dlpack_c_exchange_api__` fast exchange protocol definitions
//----------------------------------------------------------------------
/*!
* \brief Request a producer library to create a new tensor.
Expand All @@ -391,7 +391,7 @@ typedef struct DLManagedTensorVersioned {
* \param error_ctx Context for `SetError`.
* \param SetError The function to set the error.
* \return The owning DLManagedTensorVersioned* or NULL on failure.
* SetError is called exactly when NULL is returned (the implementor
* SetError is called exactly when NULL is returned (the implementer
* must ensure this).
* \note - As a C function, must not thrown C++ exceptions.
* - Error propagation via SetError to avoid any direct need
Expand Down Expand Up @@ -432,11 +432,11 @@ typedef int (*DLPackManagedTensorFromPyObjectNoSync)( //
* \brief Exports a PyObject* Tensor/NDArray to a provided DLTensor.
*
* This function provides a faster interface for temporary, non-owning, exchange.
* The producer (implementor) still owns the memory of data, strides, shape.
* The producer (implementer) still owns the memory of data, strides, shape.
* The liveness of the DLTensor and the data it views is only guaranteed until
* control is returned.
*
* This function currently assumes that the producer (implementor) can fill
* This function currently assumes that the producer (implementer) can fill
* in the DLTensor shape and strides without the need for temporary allocations.
*
* This function does not perform any stream synchronization. The consumer should query
Expand Down Expand Up @@ -488,7 +488,7 @@ typedef int (*DLPackCurrentWorkStream)( //
* \brief Imports a DLManagedTensorVersioned to a PyObject* Tensor/NDArray.
*
* Convert an owning DLManagedTensorVersioned* to the Python tensor of the
* producer (implementor) library with the correct type.
* producer (implementer) library with the correct type.
*
* This function does not perform any stream synchronization.
*
Expand Down Expand Up @@ -532,16 +532,17 @@ typedef struct DLPackExchangeAPIHeader {
* \brief Framework-specific function pointers table for DLPack exchange.
*
* Additionally to `__dlpack__()` we define a C function table sharable by
* Python implementations via `__c_dlpack_exchange_api__`.
* This attribute must be set on the type as a Python integer compatible
* with `PyLong_FromVoidPtr`/`PyLong_AsVoidPtr`.
*
* Python implementations via `__dlpack_c_exchange_api__`.
* This attribute must be set on the type as a Python PyCapsule
* with name "dlpack_exchange_api".
*
* A consumer library may use a pattern such as:
*
* \code
*
* PyObject *api_obj = type(tensor_obj).__c_dlpack_exchange_api__; // as C-code
* MyDLPackExchangeAPI *api = PyLong_AsVoidPtr(api_obj);
* PyObject *api_obj = type(tensor_obj).__dlpack_c_exchange_api__; // as C-code
Comment thread
tqchen marked this conversation as resolved.
Outdated
* MyDLPackExchangeAPI *api = PyCapsule_GetPointer(api_obj, "dlpack_exchange_api");
* if (api == NULL && PyErr_Occurred()) { goto handle_error; }
*
* \endcode
Expand Down