Skip to content

[Work in progress; do not merge] Make odh_ta_lmes_job build hermetically#562

Open
m-misiura wants to merge 1 commit into
red-hat-data-services:mainfrom
m-misiura:hermetic_build_odh_ta_lmes_job
Open

[Work in progress; do not merge] Make odh_ta_lmes_job build hermetically#562
m-misiura wants to merge 1 commit into
red-hat-data-services:mainfrom
m-misiura:hermetic_build_odh_ta_lmes_job

Conversation

@m-misiura

Copy link
Copy Markdown

Introduction

This PR attempts to deal with the following JIRA

Initial implementation

  • Dockerfile.konflux.lmes-job: single-stage build using AIPCC CPU base image, with:

    • RPM prefetch for skopeo
    • Conditional pip install (CUDA lockfile for amd64/arm64, CPU for ppc64le/s390x)
    • chmod -R og+rX to fix editable install permissions for non-root user
  • Requirements — three lockfiles compiled against AIPCC 3.5-EA2:

    • requirements-build.txt (setuptools, wheel)
    • requirements-cuda.txt (103 packages from CUDA index)
    • requirements-cpu.txt (103 packages from CPU index)
  • RPM — rpms.in.yaml + rpms.lock.yaml for skopeo across 4 arches

Assumed for the time being that tqdm-multiprocess is not needed

@m-misiura

Copy link
Copy Markdown
Author

/build-konflux

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Konflux “odh_ta_lmes_job” container build to be hermetic by switching to an AIPCC base image and introducing pre-fetched, pinned dependency inputs (RPM + pip lockfiles) to avoid resolving dependencies during the build.

Changes:

  • Replace the multi-stage Docker build with a single-stage AIPCC-based build that installs RPMs/pip deps from prefetch outputs and adjusts permissions for non-root execution.
  • Add pinned pip lockfiles (CPU/CUDA + build tooling) and RPM lock/input files (skopeo) plus a UBI repo definition for RPM prefetching.
  • Enable hermetic mode in the Tekton PipelineRun and configure prefetch-input for pip + rpm.

Reviewed changes

Copilot reviewed 9 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
Dockerfile.konflux.lmes-job Switches to AIPCC base image, installs skopeo via pre-fetched RPM repo, installs pinned pip deps, and configures runtime env/permissions.
pyproject.toml Updates Python requirement to 3.12 and bumps/pins core + optional dependency versions.
.tekton/odh-ta-lmes-job-pull-request.yaml Enables hermetic mode and adds prefetch inputs for pip requirements and RPMs.
requirements/requirements-cpu.txt UV-compiled CPU lockfile for pinned Python dependencies.
requirements/requirements-cuda.txt UV-compiled CUDA lockfile for pinned Python dependencies.
requirements/requirements-build.in Build-tool input pins for lockfile generation (setuptools/wheel).
requirements/requirements-build.txt UV-compiled build-tool lockfile.
requirements/rpms.in.yaml Declares RPM inputs (skopeo) and repo/containerfile context for lock generation.
requirements/rpms.lock.yaml Locked RPM URLs/checksums across architectures for hermetic RPM install.
ubi.repo UBI9 BaseOS/AppStream repo definitions used for RPM prefetch/locking.
.gitignore Ignores hermeto-related generated artifacts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

fi

# Install the package
RUN pip install --no-cache-dir --no-deps -e .
Comment thread pyproject.toml Outdated
Comment on lines +35 to +38
"sqlitedict==2.1.0",
"torch==2.6.0",
"tqdm-multiprocess==0.0.11",
"transformers==4.48.0",
"zstandard==0.23.0",
"dill==0.3.8",
"torch==2.11.0",
"transformers==4.57.6",
"zstandard==0.25.0",
@m-misiura m-misiura changed the title Make odh_ta_lmes_job build hermetically [Work in progress; do not merge] Make odh_ta_lmes_job build hermetically Jun 17, 2026
@m-misiura

Copy link
Copy Markdown
Author

/build-konflux

2 similar comments
@m-misiura

Copy link
Copy Markdown
Author

/build-konflux

@m-misiura

Copy link
Copy Markdown
Author

/build-konflux

@m-misiura m-misiura force-pushed the hermetic_build_odh_ta_lmes_job branch from bb2f7a2 to d5defca Compare June 30, 2026 13:50
@m-misiura m-misiura force-pushed the hermetic_build_odh_ta_lmes_job branch from d5defca to 1036f02 Compare June 30, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants