The cpython structures(e.g. Py_buffer) are excluded, compiling failed (e.g.ao/torchao/csrc/cuda/tensor_core_tiled_layout/tensor_core_tiled_layout.cu)

[codereba]

Reproduce steps:

• Prepare the enviroment

Ubuntu 2204
Python 3.10.12
torch version 2.7.0

• Clone the tag v0.8.0 of torchao:

  git clone https://github.com/pytorch/ao.git
  cd ao
  git checkout v0.8.0

• Run the command:
  python ./setup.py --verbose build

The error messages output:

lib/python3.10/site-packages/torch/include/pybind11/buffer_info.h(154): error: identifier "Py_buffer" is undefined

lib/python3.10/site-packages/torch/include/pybind11/buffer_info.h(155): error: identifier "Py_buffer" is undefined

lib/python3.10/site-packages/torch/include/pybind11/buffer_info.h(182): error: identifier "Py_buffer" is undefined

lib/python3.10/site-packages/torch/include/pybind11/buffer_info.h(119): error: more than one user-defined conversion from "<error-type>" to "std::vector<int64_t, std::allocator<int64_t>>" applies:

lib/python3.10/site-packages/torch/include/pybind11/buffer_info.h(109): error: no instance of constructor "pybind11::buffer_info::buffer_info" matches the argument list
            argument types are: (<error-type>, <error-type>, <error-type>, <error-type>, {...}, <error-type>, <error-type>)

lib/python3.10/site-packages/torch/include/pybind11/buffer_info.h(141): error: no instance of overloaded function "std::swap" matches the argument list
            argument types are: (<error-type> *, <error-type> *)

lib/python3.10/site-packages/torch/include/pybind11/buffer_info.h(149): error: identifier "PyBuffer_Release" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(263): error: identifier "PyGILState_Check" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(278): error: identifier "PyGILState_Check" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(322): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(326): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(489): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(491): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(647): error: identifier "PyTracebackObject" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(666): error: pointer to incomplete class type "PyCodeObject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(666): error: type name is not allowed

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(666): error: expected an expression

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(670): error: pointer to incomplete class type "PyCodeObject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(670): error: type name is not allowed

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(670): error: expected an expression

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(674): error: identifier "PyFrame_GetBack" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(674): error: cannot deduce "auto" type

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(947): error: identifier "PyInstanceMethod_Check" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(948): error: identifier "PyInstanceMethod_GET_FUNCTION" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(949): error: identifier "PyMethod_Check" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(950): error: identifier "PyMethod_GET_FUNCTION" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(1261): error: identifier "PyListObject" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(1261): error: expected an expression

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(1261): error: identifier "PyTupleObject" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(1261): error: expected an expression

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(1359): error: identifier "PyStaticMethod_Type" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(1777): error: identifier "PyByteArray_AS_STRING" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(1778): error: identifier "PyByteArray_GET_SIZE" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2098): error: identifier "PyTuple_GET_SIZE" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2187): error: identifier "PyList_GET_SIZE" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2269): error: identifier "PyStaticMethod_New" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2269): error: identifier "PyStaticMethod_New" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2274): error: identifier "PyObject_CheckBuffer" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2274): error: identifier "PyObject_CheckBuffer" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2277): error: identifier "PyBUF_STRIDES" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2277): error: identifier "PyBUF_FORMAT" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2279): error: identifier "PyBUF_WRITABLE" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2281): error: expected a type specifier

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2281): error: cannot deduce "auto" type

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2282): error: identifier "PyObject_GetBuffer" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2309): error: identifier "PyMemoryView_FromBuffer" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2390): error: identifier "PyBUF_READ" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2390): error: identifier "PyBUF_WRITE" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2423): error: identifier "Py_buffer" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2435): error: identifier "PyMemoryView_FromBuffer" is undefined

lib/python3.10/site-packages/torch/include/pybind11/pytypes.h(2459): error: identifier "PyObject_LengthHint" is undefined

lib/python3.10/site-packages/torch/include/pybind11/detail/internals.h(557): error: pointer to incomplete class type "_ts" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/cpp_conduit.h(24): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/cpp_conduit.h(29): error: identifier "_PyType_Lookup" is undefined

lib/python3.10/site-packages/torch/include/pybind11/detail/cpp_conduit.h(30): error: identifier "PyInstanceMethod_Check" is undefined

lib/python3.10/site-packages/torch/include/pybind11/detail/type_caster_base.h(117): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/type_caster_base.h(151): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/type_caster_base.h(161): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/type_caster_base.h(465): error: identifier "_PyThreadState_UncheckedGet" is undefined

lib/python3.10/site-packages/torch/include/pybind11/cast.h(346): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/cast.h(346): error: cannot deduce "auto" type

lib/python3.10/site-packages/torch/include/pybind11/cast.h(368): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/cast.h(411): error: identifier "PyUnicode_AsUTF8AndSize" is undefined

lib/python3.10/site-packages/torch/include/pybind11/cast.h(410): error: cannot deduce "auto" type

lib/python3.10/site-packages/torch/include/pybind11/cast.h(1819): error: identifier "PyGILState_Check" is undefined

lib/python3.10/site-packages/torch/include/pybind11/attr.h(105): error: identifier "PyHeapTypeObject" is undefined

lib/python3.10/site-packages/torch/include/pybind11/attr.h(109): error: incomplete type is not allowed

lib/python3.10/site-packages/torch/include/pybind11/attr.h(107): error: incomplete type is not allowed

lib/python3.10/site-packages/torch/include/pybind11/attr.h(314): error: incomplete type is not allowed

lib/python3.10/site-packages/torch/include/pybind11/attr.h(355): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(32): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(51): error: incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(57): error: incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(61): error: incomplete type is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(61): error: identifier "PyHeapTypeObject" is undefined

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(61): error: identifier "heap_type" is undefined

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(74): error: expected an expression

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(74): error: cannot deduce "auto" type

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(74): error: expected a ";"

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(84): error: cannot deduce "auto" type

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(141): error: identifier "_PyType_Lookup" is undefined

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(154): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(165): error: incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(176): error: identifier "_PyType_Lookup" is undefined

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(177): error: identifier "PyInstanceMethod_Check" is undefined

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(181): error: incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(188): error: incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(245): error: incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(259): error: expected an expression

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(259): error: cannot deduce "auto" type

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(259): error: expected a ";"

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(269): error: cannot deduce "auto" type

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(299): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(359): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(445): error: identifier "_PyObject_GetDictPtr" is undefined

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(469): error: pointer to incomplete class type "_typeobject" is not allowed

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(499): error: expected an expression

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(499): error: cannot deduce "auto" type

lib/python3.10/site-packages/torch/include/pybind11/detail/class.h(499): error: expected a ";"

Is the code related to this error? if Py_LIMITED_API is defined, Py_buffer should not be included:
The 3.10 code:

#ifndef Py_LIMITED_API
#  define Py_CPYTHON_OBJECT_H
#  include  "cpython/object.h"
#  undef Py_CPYTHON_OBJECT_H
#endif

The main branch of python:
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030b0000):
https://github.com/python/cpython/blob/0ff16115741aeaaaf7f963f68d5c575efb960277/Include/pybuffer.h#L9

Thank you.

[jcaip]

@codereba What torch version are you using?

I tried with torch==2.6.0 (last stable and the build succeeded for me. You might need to update your torch version.

[codereba]

@codereba What torch version are you using?

I tried with torch==2.6.0 (last stable and the build succeeded for me. You might need to update your torch version.

Hi @jcaip ,
My installed torch version is 2.7.0, because that's required by torchchat exporting function(torchchat/utils/scripts/install_et.sh), I need for building torchchat.
I will try torch 2.6.0.

Thank you.

[codereba]

@codereba What torch version are you using?

I tried with torch==2.6.0 (last stable and the build succeeded for me. You might need to update your torch version.

Hi @jcaip ,
I try torch 2.6.0, the error isn't reproduced, thank you.

I hope to find the root cause, this error maybe related to the torch header files? I found the header directories in the compiling command:
/usr/local/cuda/bin/nvcc --generate-dependencies-with-compile ... I.venv/lib/python3.10/site-packages/torch/include -I.venv/lib/python3.10/site-packages/torch/include/torch/csrc/api/include -I/usr/include/python3.10 -c -c .../ao/torchao/csrc/cuda/fp6_llm/fp6_linear.cu -o .../torchao/csrc/cuda/fp6_llm/fp6_linear.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ...