Type hint issues for enum after updating to pybind11 3.0.1

[ottmar-zittlau]

Hi,

I have some issues after updating pybind11 from v2.13.6 to v3.0.1.

I tried to reduce everything to a small example.
I'm using conan to get pybind11.

My python requirements are contained in requirements.txt

conan==2.22.2
pybind11_stubgen==2.5.5

My conan requirements are contained in conanfile.txt

[requires]
pybind11/3.0.1

[layout]
cmake_layout

[generators]
CMakeDeps
CMakeToolchain

My CMakeLists.txt reads

cmake_minimum_required(VERSION 3.31)
project(pybind11test CXX)

find_package(pybind11)

pybind11_add_module(pybind11test pybind11test.cpp)

And my source file pybind11test.cpp is

#include <pybind11/pybind11.h>
#include <pybind11/native_enum.h>
#include <pybind11/stl.h>

namespace py = pybind11;
using namespace py::literals;

enum class Enum1
{
     VALUE1 = 0,
     VALUE2 = 1
};

struct Struct
{
     size_t value = 100;
     std::optional<Enum1> enum1{};
};

PYBIND11_MODULE(pybind11test, pybind11test_module)
{
     py::native_enum<Enum1>(pybind11test_module, "Enum1", "enum.Enum")
         .value("VALUE1", Enum1::VALUE1)
         .value("VALUE2", Enum1::VALUE2)
         .finalize();

     // py::enum_<Enum1>(pybind11test_module, "Enum1")
     //     .value("VALUE1", Enum1::VALUE1)
     //     .value("VALUE2", Enum1::VALUE2);

     py::class_<Struct>(pybind11test_module, "Struct")
         .def(py::init<>())
         .def(py::init<size_t>(), "Struct", "value"_a)
         .def(py::init<size_t, Enum1>(), "Struct", "value"_a, "enum1"_a)
         .def_readwrite("value", &Struct::value)
         .def_readwrite("enum1", &Struct::enum1);
}

If I run the following commands:

pip install -r requirements.txt
conan install .
cmake --preset conan-release
cmake --build --preset conan-release
PYTHONPATH=./build/Release/ pybind11-stubgen pybind11test

I get the following stubs file:

from __future__ import annotations
import enum
import typing
__all__: list[str] = ['Enum1', 'Struct']
class Enum1(enum.Enum):
    VALUE1: typing.ClassVar[Enum1]  # value = <Enum1.VALUE1: 0>
    VALUE2: typing.ClassVar[Enum1]  # value = <Enum1.VALUE2: 1>
class Struct:
    enum1: pybind11test.Enum1 | None
    @typing.overload
    def __init__(self) -> None:
        ...
    @typing.overload
    def __init__(self, value: typing.SupportsInt) -> None:
        """
        Struct
        """
    @typing.overload
    def __init__(self, value: typing.SupportsInt, enum1: Enum1) -> None:
        """
        Struct
        """
    @property
    def value(self) -> int:
        ...
    @value.setter
    def value(self, arg0: typing.SupportsInt) -> None:
        ...

My issue is the that member variable enum1 of struct Struct is getting type pybind11test.Enum1 - which mypy complains about. The argument to the constructor in Struct gets the type Enum1 which seems to be okay.

If I run everything which the previous pybind11 version (change version in conanfile.txt and use the old enum-wrapping mechanism) I get:

from __future__ import annotations
import typing
__all__: list[str] = ['Enum1', 'Struct']
class Enum1:
    """
    Members:
    
      VALUE1
    
      VALUE2
    """
    VALUE1: typing.ClassVar[Enum1]  # value = <Enum1.VALUE1: 0>
    VALUE2: typing.ClassVar[Enum1]  # value = <Enum1.VALUE2: 1>
    __members__: typing.ClassVar[dict[str, Enum1]]  # value = {'VALUE1': <Enum1.VALUE1: 0>, 'VALUE2': <Enum1.VALUE2: 1>}
    def __eq__(self, other: typing.Any) -> bool:
        ...
    def __getstate__(self) -> int:
        ...
    def __hash__(self) -> int:
        ...
    def __index__(self) -> int:
        ...
    def __init__(self, value: int) -> None:
        ...
    def __int__(self) -> int:
        ...
    def __ne__(self, other: typing.Any) -> bool:
        ...
    def __repr__(self) -> str:
        ...
    def __setstate__(self, state: int) -> None:
        ...
    def __str__(self) -> str:
        ...
    @property
    def name(self) -> str:
        ...
    @property
    def value(self) -> int:
        ...
class Struct:
    enum1: Enum1 | None
    value: int
    @typing.overload
    def __init__(self) -> None:
        ...
    @typing.overload
    def __init__(self, value: int) -> None:
        """
        Struct
        """
    @typing.overload
    def __init__(self, value: int, enum1: Enum1) -> None:
        """
        Struct
        """

Here the enum contains more methods, but the types in the struct are more consistent (in my opinion) and also mypy doesn't complain.
Is there a way to configure stubgen so that it uses the shorter type hint for both function arguments and member variables? This would help me here.

If I use the new pybind11 version with the old enum_-mechanism I get the old stubs except for the type of the member variable - this remains pybind11test.Enum1.

I also get another mypy-error with the new type hints:

Detected enum "pybind11test.Enum1" in a type stub with zero members. There is a chance this is due to a recent change in the semantics of enum membership. If so, use member = value to mark an enum member, instead of member: type

Mypy points me to https://typing.python.org/en/latest/spec/enums.html#defining-members - maybe this could also be of interest.

Thanks and best regards
oz