Skip to content

Commit fcc727f

Browse files
aaronj0guitargeek
authored andcommitted
[CPyCppyy] Drop __array__ from std::vector pythonizations
The addition of the __array__ utility to std::vector Python proxies causes a bug where the resulting array is a single dimension, causing loss of data when converting to numpy arrays, for >1dim vectors. The recursive nature of this function, passes each subarray (pydata) to the next call and only the final buffer is cast to a lowlevel view and resized (in VectorData), resulting in only the first 1D array to be returned. See #17729 Since this C++ pythonization was added with the upgrade in 6.32, and is only defined and used recursively, the safe option is to disable this function and no longer add it. It is temporarily removed to prevent errors due to -Wunused-function
1 parent e555e31 commit fcc727f

File tree

3 files changed

+81
-1
lines changed

3 files changed

+81
-1
lines changed

bindings/pyroot/cppyy/CPyCppyy/src/Pythonize.cxx

+15-1
Original file line numberDiff line numberDiff line change
@@ -527,6 +527,13 @@ PyObject* VectorData(PyObject* self, PyObject*)
527527
}
528528

529529

530+
// This function implements __array__, added to std::vector python proxies and causes
531+
// a bug (see explanation at Utility::AddToClass(pyclass, "__array__"...) in CPyCppyy::Pythonize)
532+
// The recursive nature of this function, passes each subarray (pydata) to the next call and only
533+
// the final buffer is cast to a lowlevel view and resized (in VectorData), resulting in only the
534+
// first 1D array to be returned. See https://github.com/root-project/root/issues/17729
535+
// It is temporarily removed to prevent errors due to -Wunused-function, since it is no longer added.
536+
#if 0
530537
//---------------------------------------------------------------------------
531538
PyObject* VectorArray(PyObject* self, PyObject* args, PyObject* kwargs)
532539
{
@@ -537,7 +544,7 @@ PyObject* VectorArray(PyObject* self, PyObject* args, PyObject* kwargs)
537544
Py_DECREF(pydata);
538545
return newarr;
539546
}
540-
547+
#endif
541548

542549
//-----------------------------------------------------------------------------
543550
static PyObject* vector_iter(PyObject* v) {
@@ -1810,8 +1817,15 @@ bool CPyCppyy::Pythonize(PyObject* pyclass, const std::string& name)
18101817
Utility::AddToClass(pyclass, "__real_data", "data");
18111818
Utility::AddToClass(pyclass, "data", (PyCFunction)VectorData);
18121819

1820+
// The addition of the __array__ utility to std::vector Python proxies causes a
1821+
// bug where the resulting array is a single dimension, causing loss of data when
1822+
// converting to numpy arrays, for >1dim vectors. Since this C++ pythonization
1823+
// was added with the upgrade in 6.32, and is only defined and used recursively,
1824+
// the safe option is to disable this function and no longer add it.
1825+
#if 0
18131826
// numpy array conversion
18141827
Utility::AddToClass(pyclass, "__array__", (PyCFunction)VectorArray, METH_VARARGS | METH_KEYWORDS /* unused */);
1828+
#endif
18151829

18161830
// checked getitem
18171831
if (HasAttrDirect(pyclass, PyStrings::gLen)) {
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
From 82295e09c77ae61e2fd8356be792d61addf2c801 Mon Sep 17 00:00:00 2001
2+
From: Aaron Jomy <[email protected]>
3+
Date: Mon, 31 Mar 2025 14:27:32 +0200
4+
Subject: [PATCH] [CPyCppyy] Drop `__array__` from std::vector pythonizations
5+
6+
The addition of the __array__ utility to std::vector Python proxies causes a
7+
bug where the resulting array is a single dimension, causing loss of data when
8+
converting to numpy arrays, for >1dim vectors. The recursive nature of this
9+
function, passes each subarray (pydata) to the next call and only the final
10+
buffer is cast to a lowlevel view and resized (in VectorData), resulting in
11+
only the first 1D array to be returned. See https://github.com/root-project/root/issues/17729
12+
13+
Since this C++ pythonization was added with the upgrade in 6.32, and is only
14+
defined and used recursively, the safe option is to disable this function and
15+
no longer add it. It is temporarily removed to prevent errors due to -Wunused-function
16+
---
17+
bindings/pyroot/cppyy/CPyCppyy/src/Pythonize.cxx | 16 +++++++++++++++-
18+
1 file changed, 15 insertions(+), 1 deletion(-)
19+
20+
diff --git a/bindings/pyroot/cppyy/CPyCppyy/src/Pythonize.cxx b/bindings/pyroot/cppyy/CPyCppyy/src/Pythonize.cxx
21+
index 9b87905bab..0510c1c6ac 100644
22+
--- a/bindings/pyroot/cppyy/CPyCppyy/src/Pythonize.cxx
23+
+++ b/bindings/pyroot/cppyy/CPyCppyy/src/Pythonize.cxx
24+
@@ -527,6 +527,13 @@ PyObject* VectorData(PyObject* self, PyObject*)
25+
}
26+
27+
28+
+// This function implements __array__, added to std::vector python proxies and causes
29+
+// a bug (see explanation at Utility::AddToClass(pyclass, "__array__"...) in CPyCppyy::Pythonize)
30+
+// The recursive nature of this function, passes each subarray (pydata) to the next call and only
31+
+// the final buffer is cast to a lowlevel view and resized (in VectorData), resulting in only the
32+
+// first 1D array to be returned. See https://github.com/root-project/root/issues/17729
33+
+// It is temporarily removed to prevent errors due to -Wunused-function, since it is no longer added.
34+
+#if 0
35+
//---------------------------------------------------------------------------
36+
PyObject* VectorArray(PyObject* self, PyObject* args, PyObject* kwargs)
37+
{
38+
@@ -537,7 +544,7 @@ PyObject* VectorArray(PyObject* self, PyObject* args, PyObject* kwargs)
39+
Py_DECREF(pydata);
40+
return newarr;
41+
}
42+
-
43+
+#endif
44+
45+
//-----------------------------------------------------------------------------
46+
static PyObject* vector_iter(PyObject* v) {
47+
@@ -1810,8 +1817,15 @@ bool CPyCppyy::Pythonize(PyObject* pyclass, const std::string& name)
48+
Utility::AddToClass(pyclass, "__real_data", "data");
49+
Utility::AddToClass(pyclass, "data", (PyCFunction)VectorData);
50+
51+
+ // The addition of the __array__ utility to std::vector Python proxies causes a
52+
+ // bug where the resulting array is a single dimension, causing loss of data when
53+
+ // converting to numpy arrays, for >1dim vectors. Since this C++ pythonization
54+
+ // was added with the upgrade in 6.32, and is only defined and used recursively,
55+
+ // the safe option is to disable this function and no longer add it.
56+
+#if 0
57+
// numpy array conversion
58+
Utility::AddToClass(pyclass, "__array__", (PyCFunction)VectorArray, METH_VARARGS | METH_KEYWORDS /* unused */);
59+
+#endif
60+
61+
// checked getitem
62+
if (HasAttrDirect(pyclass, PyStrings::gLen)) {
63+
--
64+
2.43.0
65+

bindings/pyroot/cppyy/sync-upstream

+1
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ git apply patches/CPyCppyy-Prevent-construction-of-agg-init-for-tuple.patch
5959
git apply patches/CPyCppyy-Support-conversion-from-str-to-char.patch # https://github.com/wlav/CPyCppyy/pull/21
6060
git apply patches/CPyCppyy-Perform-function-style-casts.patch # https://github.com/wlav/CPyCppyy/pull/34
6161
git apply patches/CPyCppyy-Use-PyMapping_GetOptionalItemString-where-necessary.patch # https://github.com/wlav/CPyCppyy/pull/44
62+
git apply patches/CPyCppyy-Disable_std-vector-numpy-array-pythonization.patch
6263
git apply patches/cppyy-No-CppyyLegacy-namespace.patch
6364
git apply patches/cppyy-Remove-Windows-workaround.patch
6465
git apply patches/cppyy-Don-t-enable-cling-autoloading.patch

0 commit comments

Comments
 (0)