Error installing solarfactors on Py3.12 due to Shapely pinned versions

[echedey-ls]

Describe the bug
I can't install solarfactors on Ubuntu 22.04 LTS.

To Reproduce
Steps to reproduce the behavior:

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.12 python3.12-venv
python3.12 -m pip install solarfactors

Output:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/lib/python3/dist-packages/pip/__main__.py", line 29, in <module>
    from pip._internal.cli.main import main as _main
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/main.py", line 9, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/main_parser.py", line 8, in <module>
    from pip._internal.cli import cmdoptions
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/cmdoptions.py", line 23, in <module>
    from pip._internal.cli.parser import ConfigOptionParser
  File "/usr/lib/python3/dist-packages/pip/_internal/cli/parser.py", line 12, in <module>
    from pip._internal.configuration import Configuration, ConfigurationError
  File "/usr/lib/python3/dist-packages/pip/_internal/configuration.py", line 26, in <module>
    from pip._internal.utils.logging import getLogger
  File "/usr/lib/python3/dist-packages/pip/_internal/utils/logging.py", line 27, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "/usr/lib/python3/dist-packages/pip/_internal/utils/misc.py", line 39, in <module>
    from pip._internal.locations import get_major_minor_version
  File "/usr/lib/python3/dist-packages/pip/_internal/locations/__init__.py", line 14, in <module>
    from . import _distutils, _sysconfig
  File "/usr/lib/python3/dist-packages/pip/_internal/locations/_distutils.py", line 9, in <module>
    from distutils.cmd import Command as DistutilsCommand
ModuleNotFoundError: No module named 'distutils'

Expected behavior
solarfactors to be installed correctly on Python 3.12

Screenshots
N/A

Versions:

• pvlib.__version__: N/A
• pandas.__version__: N/A
• python: 3.12

Additional context
Versioneer issue for Py3.12, version giving the error matches some comment: python-versioneer/python-versioneer#375
Versioneer updating guidelines: https://github.com/python-versioneer/python-versioneer/blob/master/UPGRADING.md

User error: https://groups.google.com/g/pvlib-python/c/0h-h7LBCvtQ

I found the error

It is due to the pinned version of shapely. Shapely <2 does limit the setuptools version to <63, which relies in features removed in Py3.12

[markcampanelli]

@echedey-ls As I mentioned elsewhere, setuptools-scm has served me well for simplifying and streamlining Python versioning in a pyproject.toml file for package configuration. It is even mentioned by the Python Packaging Authority for setuptools-based builds that use SCM:

https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html .

If you desire to take this on, then I’d be happy to advise further and review.

See also: NREL/PVDegradationTools#37

[echedey-ls]

Yes, I completely agree with you. It makes much more sense to use the same packaging infrastructure all along the pvlib organization - that is using setuptools-scm and dropping versioneer among other changes this repo would need. Right now I'm pursuing that path, thou it isn't going exactly well.

I've seen the pvdeg package, but it's pinned to Python versions lower than 3.12. This specific problem I'm trying to address is specific to python 3.12.

[echedey-ls]

You're gonna laugh at me: the problem is that latest shapely before v2 is 1.8.5, and that one does NOT support Py3.12. Source: https://pypi.org/project/shapely/1.8.5.post1/#files

The least Pythonic Python moment be like :(

EDIT: @kandersolar , may I push #4 forward? It would also help my PR at pvlib/pvlib-python#2106

[markcampanelli]

You're gonna laugh at me: the problem is that latest shapely before v2 is 1.8.5, and that one does NOT support Py3.12. Source: https://pypi.org/project/shapely/1.8.5.post1/#files

@echedey-ls People may call me paranoid, but this is why I don't promise compatibility of pvfit with future Python releases: https://github.com/markcampanelli/pvfit/blob/v0.0.1/pyproject.toml#L15

It's pretty hard to predict the future of complex systems :).

[kandersolar]

Yep, shapely is the issue. Sorry @echedey-ls you had to figure it out on your own -- I should have opened an issue documenting it long ago :)

Help on #4 would certainly be welcome, but beware -- it is no small task. I've tried several times to make progress on that PR, but I get lost every time and say "some day when I have a week of free time, I will work on this".

I think at this point we should try to break down the broader "get rid of shapely" task into smaller pieces and then work on them one PR at a time, instead of trying to do it all in one big PR. If you are interested in helping with this, that would be great!

[k10blogger]

A pretty noob question here.
What is the challenge in bumping the shapely version to 2.0.5?
If we keep aside the removal of shapely completely what stops for pinning the requirement to shapely version 2.0.5.
I have just added a 3.12 python test matrix to my workspace and this seems to be breaking for me.

[kandersolar]

Shapely 2.0 changed many parts of its API, meaning that code written for Shapely 1.* does not work with Shapely 2.*. So we cannot simply update the allowed version of shapely--perhaps the installation would succeed, but trying to use it would just result in errors. See also SunPower/pvfactors#126

There are two ways forward here: either the code needs to be rewritten to be compatible with 2.*, or the dependency on shapely needs to be removed entirely.

[joseph-sch]

I'm working on a PR to migrate solarfactors to Shapely >= 2.0: main...solargik:solarfactors:migrate_to_shapely_2.0

So far, it has involved replacing inheritance by composition for classes that we derived from Shapely classes, but it seems this can be done without changing the API of these classes.

I still need to check the warnings and benchmark before/after to make sure performance is not impacted negatively.

Provided this works, it will allow Python 3.12, NumPy 2.0, etc.

I've also refreshed the README file (links, etc.)

The pvfactors license is permissive but what should we do about the name (pvfactors/solarfactors) if we make substantive changes? @markcampanelli @campanelli-sunpower @kandersolar maybe?

[joseph-sch]

The PR #22 is ready for review.

[markcampanelli]

The pvfactors license is permissive but what should we do about the name (pvfactors/solarfactors) if we make substantive changes?

@joseph-sch My understanding (which may be incorrect/incomplete) is that the fork from pvfactors to solarfactors cleared the way for the code to further evolve without restriction. I presume that the license implications for code changes going forward were properly considered when the fork happened.