There are no eigen specific features being used other than easy io with numpy via pybind11. Nanobind has it's on numpy compatible class so it could replace pybind11.
Potential performance improvements
Remove dependency on eigen
https://github.com/wjakob/nanobind