Challenges in latest Ubuntu (June 2025)

[jimallman]

Our AWS servers are currently running Ubuntu 24.04.2 LTS (noble). This includes Python 3.12 by default, which presents a couple of problems for our existing Ansible playbooks. So far the remedies seem clunky (or are blocked by one of our package managers), so I'm hoping for some opinions on the best path forward. Here are the blockers I've found so far:

---

anyjson. Some of our subsystems (peyotl, phylesystem-api, probably others) are still using the deprecated anyjson module, which is incompatible with the current setuptools.

This was handled previously by using pip to downgrade setuptools (briefly) to install just this module, then restoring(?) the newer version for the remainder of deployment. This no longer works as we install python3-setuptools via apt, so pip can no longer manage setuptools.

Possible remedies:

• Move all setuptools management to pip, instead of apt. (Can we install pip w/o this prior installation?)
• Force the installation of a newer python3-setuptools (which contains a fix), presumably by adding another apt repo.
• Sweep all of our Python code to remove any dependency on anyjson
• Install python3-anyjson via apt, and remove it from all requirements.txt files (eg, in peyotl).

---

ImpImporter. Pyramid's pserve fails to launch due to some dependency's use of the deprecated pkgutil.ImpImporter (removed completely in Python3.12).

Possible remedies:

• Install python3.11 (or other pre-3.12 version) for our apps and virtual environments, presumably by adding another apt repo.
• Search for and update (or replace) any python dependencies that use ImpImporter. TODO: Try clearing the current pyramid_venv and re-installing with the latest version of everything (requirements.in instead of requirements.txt)?
• Try downloading and invoking the ensurepip script, which does and end run around our package managers, but might fix this issue. I think we'd need to use this instead of our apt-installed pip.

---

So far I've been working on my pyramid-webapps branch based on main. Should I be paying closer attention to the development or ubuntu24.04 branches of ot-ansible? So far I don't see answers there.

Of the suggested kludges above, which do you find the least offensive? i.e.

• Using special tools like ensurepip alongside our existing package managers (apt, pip)
• Adding apt repositories (temporarily?) to install non-standard versions
• Removing apt-managed requirements from our app-specific requirements.* files

[snacktavish]

Another potentially klunky option is to just set up the AWS machine to run an older version of Ubuntu... but that doesn't seem very future-proof.

Honestly I think any of your three options at the end sound OK to me!

[jimallman]

Copying these notes from our latest Zoom call...

Discussion of Ansible logjam (conflicts between Python3.12 and our various package managers) and how to best resolve them. Some decisions:

• We should purge any remaining use of the python anyjson module
• We should try for Python3.12 support where possible, esp. for peyotl and other common dependencies.
• We should try to consolidate all python reqs in pip and requirements.* files (vs fetching some python packages from apt).
• It’s OK to add apt repos as needed for special requirements
  KEEP these repos if we can guarantee that the default repo is always
  checked first; else dispose of extra repos in Ansible, immediately after use
• Let’s clarify the status of ot-ansible development branch and our desired workflow for this repo.