META: Python 3.12 package status

[th0ma7]

Python 3.12 package status

checkmark: ✔️
xmark: ❌️

Packages formally using python 3.x

PACKAGE	Build	Install.	Working	Published	COMMENT
bazarr					needs numpy (and TC_GCC > 5)
beets					#6447
borgbackup					#6444
deluge					#6423
domoticz
duplicity
ffsync					#6429
fishnet
flexget					#6427
haproxy
homeassistant	✔️	✔️			#6453 2025.1.4 fails to cross compile some wheels for python312 - those are installed from index and not included in the package: av==13.1.0 awscrt==0.23.5 deebot_client==11.0.0 numpy==2.2.0 pandas==2.2.3 pillow==11.0.0 => use cross/pillow_ha instead
mercurial					#6422
octoprint
plexpy-custom					Ready for migration but BUG with the package install, to remove?
rdiff-backup					#6470
rutorrent					#6404 update to python312 pending
sabnzbd					#6431
sickchill
salt-master
salt-minion
tvheadend					#6424
znc					#6425

Framework clean-up

• dynamic crossenv + python3*-wheels + python310-313 updates #6282
• wheel.mk: Migrate to using status cookie #6389
• wheel building: Pile of small fixes to recent changes #6409
• Add wheel top-level status cookie and mark armv5 as unsupported >= python 3.12 #6428
• Review content of crossenv to respect build vs cross. #6437
• [WIP] Handle meson based python-wheel #6454

[th0ma7]

@hgy59 @mreid-tt @SynoCommunity/developers Considering the following, strongly suggesting to migrate to python 3.12 which would be a lot less impactful and wait anothe year or so before switching to a newer python version considering impacts on older DSM6 archs.

python311:

• newer pillow now finally compile on all archs!!!

python312:

• newer numpy finally compile to the detriment that older compatible with gcc-4.9.x no longer works (now DSM7 only)
• armv5 no longer supported
• impacted packages: bazarr, homeassistant

python313:

• greenlet only work with newer gcc (now DSM7 only)
• armv5 no longer supported
• impacted packages: bazarr, ffsync, flexget, homeassistant, sickchill

[th0ma7]

I've added a first set of relatively "easy" wins for early testing: deluge, mercurial, tvheadend, znc

[mreid-tt]

Are we considering a big-bang approach where we prepare all the packages for Python 3.1.2 and release them simultaneously? Or should we prioritize quick wins, releasing simpler packages first and gradually tackling the more complex ones?

Regarding the current FlexGet PR I have, should I proceed with that change now, or wait until all the packages are ready for release? The main advantage of a big-bang approach is that users would only need to have both Python 3.1.1 and 3.1.2 installed temporarily, rather than having to maintain them for an extended period. However, this approach could introduce more risk and delays, especially with more complex packages. I’d appreciate your thoughts on this.

[th0ma7]

Are we considering a big-bang approach where we prepare all the packages for Python 3.1.2 and release them simultaneously? Or should we prioritize quick wins, releasing simpler packages first and gradually tackling the more complex ones?

no I'm not, I'd rather have a few easy wins first, like the one i started with.

Regarding the current FlexGet PR I have, should I proceed with that change now, or wait until all the packages are ready for release? The main advantage of a big-bang approach is that users would only need to have both Python 3.1.1 and 3.1.2 installed temporarily, rather than having to maintain them for an extended period. However, this approach could introduce more risk and delays, especially with more complex packages. I’d appreciate your thoughts on this.

i would be tempted to suggest you release as-is first using python 3.11, and in a week or two once issues if any have been found with your 3.11 release (or the easy wins above) you migrate "as-is" to py312. That's why would recommend.

[hgy59]

@th0ma7 I have taken the octoprint package.
It successfully builds the current and the updated version (1.10.1 and 1.10.3) with python312 but it does not run. There is no error shown, it just terminates and the pid file is gone.
I have the same issue with python311 (only the already published package runs).

Tested on VitualDSM 7.2.2.

[mreid-tt]

@th0ma7, thanks for the feedback. I'll start with those I've touched before flexget, ffsync and then perhaps look at sabnzbd, bazarr and move from there.

[th0ma7]

@th0ma7 I have taken the octoprint package. It successfully builds the current and the updated version (1.10.1 and 1.10.3) with python312 but it does not run. There is no error shown, it just terminates and the pid file is gone. I have the same issue with python311 (only the already published package runs).

Tested on VitualDSM 7.2.2.

I just tested with deluge PR on my armv7 NAS. First off python runs ok, basic testing showed it being functional. When installing deluge this came into the logs:

2025/01/26 15:19:28     ERROR: Could not find a version that satisfies the requirement libtorrent==2.0.10 (from versions: none)
2025/01/26 15:19:28     ERROR: No matching distribution found for libtorrent==2.0.10
2025/01/26 15:19:30     ERROR: Python package installation failed

That can only mean one thing: it either failed to build on github or failed to copy it over to the wheelhouse directory....

looking further into the github logs, it was built sucessfully:

2025-01-26T17:22:49.7727317Z Collecting git+https://github.com/arvidn/libtorrent.git@v2.0.10
2025-01-26T17:22:49.7734237Z   Cloning https://github.com/arvidn/libtorrent.git (to revision v2.0.10) to /tmp/pip-req-build-oyf8bquz
2025-01-26T17:22:49.7754763Z   Running command git clone --filter=blob:none --quiet https://github.com/arvidn/libtorrent.git /tmp/pip-req-build-oyf8bquz
2025-01-26T17:22:51.6474808Z   Running command git checkout -q 74bc93a37a5e31c78f0aa02037a68fb9ac5deb41
2025-01-26T17:22:52.0212517Z   Resolved https://github.com/arvidn/libtorrent.git to commit 74bc93a37a5e31c78f0aa02037a68fb9ac5deb41
2025-01-26T17:22:52.0215494Z   Running command git submodule update --init --recursive -q
2025-01-26T17:22:52.5070351Z   Preparing metadata (setup.py): started
2025-01-26T17:22:52.9431697Z   Preparing metadata (setup.py): finished with status 'done'
2025-01-26T17:22:52.9461118Z Building wheels for collected packages: libtorrent
2025-01-26T17:22:52.9487882Z   Building wheel for libtorrent (setup.py): started
2025-01-26T17:23:53.1898128Z   Building wheel for libtorrent (setup.py): still running...
2025-01-26T17:24:53.6521239Z   Building wheel for libtorrent (setup.py): still running...
2025-01-26T17:25:53.7688651Z   Building wheel for libtorrent (setup.py): still running...
2025-01-26T17:26:53.9019848Z   Building wheel for libtorrent (setup.py): still running...
2025-01-26T17:27:00.7920473Z   Building wheel for libtorrent (setup.py): finished with status 'done'
2025-01-26T17:27:00.7981113Z   Created wheel for libtorrent: filename=libtorrent-2.0.10-cp311-cp311-linux_arm.whl size=5034100 sha256=77e258ce31e09bddb25ab91caca247550f86f67f41d6c698246a131a74122301
2025-01-26T17:27:00.7982868Z   Stored in directory: /tmp/pip-ephem-wheel-cache-7rpjsi4i/wheels/67/90/66/613c0360108c8f1fabe47bc78d60c59f6520ebb48ae96c8592
2025-01-26T17:27:00.8002042Z Successfully built libtorrent
2025-01-26T17:27:00.9452691Z ===>  Installing wheel [libtorrent], version [2.0.10], type [crossenv]
2025-01-26T17:27:00.9453510Z ===>  Adding libtorrent==2.0.10 to wheelhouse/requirements-crossenv.txt

So is the file there or not? indeed it is!

root@DS115j-armv7:/var/packages/deluge/target/share/wheelhouse# ls -la /var/packages/deluge/target/share/wheelhouse/
total 10116
drwxr-xr-x 2 sc-deluge synocommunity    4096 Jan 26 12:27 .
drwxr-xr-x 3 sc-deluge synocommunity    4096 Jan 26 12:27 ..
-rw-r--r-- 1 sc-deluge synocommunity  188352 Jan 26 12:27 cffi-1.17.1-cp311-cp311-linux_armv7l.whl
-rw-r--r-- 1 sc-deluge synocommunity 1624032 Jan 26 12:27 cryptography-44.0.0-cp37-abi3-linux_armv7l.whl
-rw-r--r-- 1 sc-deluge synocommunity 3146827 Jan 26 12:27 deluge-2.1.1.dev127-py3-none-any.whl
-rw-r--r-- 1 sc-deluge synocommunity   30302 Jan 26 12:27 GeoIP-1.3.2-cp311-cp311-linux_armv7l.whl
-rw-r--r-- 1 sc-deluge synocommunity 5034100 Jan 26 12:27 libtorrent-2.0.10-cp311-cp311-linux_armv7l.whl
-rw-r--r-- 1 sc-deluge synocommunity   14496 Jan 26 12:27 MarkupSafe-3.0.2-cp311-cp311-linux_armv7l.whl
-rw-r--r-- 1 sc-deluge synocommunity   65767 Jan 26 12:27 rencode-1.0.6-cp311-cp311-linux_armv7l.whl
-rw-r--r-- 1 sc-deluge synocommunity      15 Jan 26 12:27 requirements-abi3.txt
-rw-r--r-- 1 sc-deluge synocommunity     125 Jan 26 12:27 requirements-crossenv.txt
-rw-r--r-- 1 sc-deluge synocommunity     276 Jan 26 12:27 requirements-pure.txt
-rw-r--r-- 1 sc-deluge synocommunity     416 Jan 26 12:27 requirements.txt
-rw-r--r-- 1 sc-deluge synocommunity  211536 Jan 26 12:27 zope.interface-7.2-cp311-cp311-linux_armv7l.whl

So why wasn't it able to install it? Something is odd, maybe you're hiting a similar issue? may just be with newer pip?

EDIT: Getting the exact same error on my x64 nas.

EDIT2: Was missing PYTHON_PACKAGE variable defined with python312 which I'm pretty was the issue. Tested on both x64 and armv7 and fully functional. Will now test with the reverting of the #egg= changes.

[mreid-tt]

Hmm, I've just completed a test build of FlexGet on DSM 6 in #6427 and everything seemed to install and upgrade fine. I'm attaching the install and upgrade logs to the PR for your review if it would help.

[th0ma7]

@mreid-tt that's great news! In the last round of wheel build update I didn't change "anything" in how builds are being made, I just changed how the makefiles are being processed.

That being said I did one change that I reminded: removal of the #egg= portion of URL based wheels set for deprecation in future pip releases. The wheel that failed to install is a URL type wheel, so that may only be that, to be confirmed.

EDIT: I just noticed I had not updated the PYTHON_PACKAGE variable... thus libtorrent wheel was advertising a python311 version (actually all cross-compiled wheels which failed to install, starting with libtorrent) libtorrent-2.0.10-cp311-cp311-linux_armv7l.whl