Skip to content

Problem: Archivematica's Python dependency and environment management is fragmented #1788

@replaceafill

Description

@replaceafill

Please describe the problem you'd like to be solved

  • The main application repositories and shared Python libraries commit requirements*.txt files generated by pip-compile, and most of those files are resolved with Python 3.10.
  • The Docker images for Archivematica, Storage Service, and the acceptance tests build Python with pyenv and then install dependencies with pip.
  • The Ansible source-install role creates and destroys virtualenvs, bootstraps pip and pip-tools, runs pip-sync, and adds the Deadsnakes PPA on Ubuntu even though there is no equivalent pattern for Rocky or the other RPM-based targets.
  • The DEB and RPM package specs add another split: the DEB builds use dh-virtualenv with the distro python3, while the RPM builds explicitly create virtualenvs with python3.12.

As a result, interpreter selection, dependency resolution, and operational behavior vary depending on whether the software is being developed locally, built in Docker, provisioned with Ansible, or installed from DEB or RPM packages. This keeps the ecosystem harder to maintain than it needs to be, increases drift between repos, and makes it difficult to adopt newer Python features and improvements consistently across Archivematica and its related projects.

Describe the solution you'd like to see implemented

  • Standardize the Python packaging and environment story on uv across the Archivematica repositories, dependency repositories, and the infrastructure that builds and installs them.
  • pyproject.toml should remain the dependency source of truth for Python projects, with uv.lock committed for reproducible resolution.
  • Development, CI, and Docker builds should use uv sync, uv run, and uv-managed Python instead of pip-tools, pip-sync, and pyenv.
  • The Ansible role should install uv and use it to create and populate managed environments rather than relying on distro-specific Python bootstrapping and pip-managed virtualenvs.
  • DEB and RPM build pipelines should consume the same locked dependency set.

The end state should be one documented, lockfile-based workflow that works the same way for local development, CI, containers, source installs, and package builds.

Describe alternatives you've considered

Keep the current pip/pip-tools/virtualenv model and keep improving it gradually.

Additional context

Artefactual repositories already using uv:

  • AIPscan
  • a3m
  • ansible-aipscan

These repositories already use uv.lock, uv-based CI, uv-based Docker builds, and uv-managed virtual environments, and they can serve as the migration model for the rest of the ecosystem.


For Artefactual use:

Before you close this issue, you must check off the following:

  • All pull requests related to this issue are properly linked
  • All pull requests related to this issue have been merged
  • A testing plan for this issue has been implemented and passed (testing plan information should be included in the issue body or comments)
  • Documentation regarding this issue has been written and merged
  • Details about this issue have been added to the release notes

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions