Merge RHDS main to ODH main #74
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| { | ||
| "$schema": "https://docs.renovatebot.com/renovate-schema.json", | ||
| "extends": [ | ||
| "github>red-hat-data-services/konflux-central//renovate/default-renovate.json5" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| apiVersion: tekton.dev/v1 | ||
| kind: PipelineRun | ||
| metadata: | ||
| annotations: | ||
| build.appstudio.openshift.io/repo: https://github.com/red-hat-data-services/pipelines-components?rev={{revision}} | ||
| build.appstudio.redhat.com/commit_sha: '{{revision}}' | ||
| build.appstudio.redhat.com/target_branch: '{{target_branch}}' | ||
| build.appstudio.redhat.com/pull_request_number: "{{pull_request_number}}" | ||
| pipelinesascode.tekton.dev/max-keep-runs: "3" | ||
| pipelinesascode.tekton.dev/on-comment: "^/build-konflux-automl" | ||
| pipelinesascode.tekton.dev/on-label: "[kfbuild-all, kfbuild-automl]" | ||
| pipelinesascode.tekton.dev/on-target-branch: "[{{target_branch}}]" | ||
| pipelinesascode.tekton.dev/on-event: "[pull_request]" | ||
| pipelinesascode.tekton.dev/cancel-in-progress: "true" | ||
| labels: | ||
| appstudio.openshift.io/application: automation | ||
| appstudio.openshift.io/component: pull-request-pipelines-odh-automl | ||
| pipelines.appstudio.openshift.io/type: build | ||
| name: odh-automl-on-pull-request | ||
| namespace: rhoai-tenant | ||
| spec: | ||
| timeouts: | ||
| pipeline: 8h | ||
| params: | ||
| - name: git-url | ||
| value: '{{source_url}}' | ||
| - name: revision | ||
| value: '{{revision}}' | ||
| - name: additional-tags | ||
| value: | ||
| - 'pr-{{pull_request_number}}-into-{{target_branch}}' | ||
| - name: additional-labels | ||
| value: | ||
| - version=on-pr-{{revision}} | ||
| - io.openshift.tags=odh-automl | ||
| - name: output-image | ||
| value: quay.io/rhoai/pull-request-pipelines:odh-automl-{{revision}} | ||
| - name: build-platforms | ||
| value: | ||
| - linux/x86_64 | ||
| - linux/ppc64le | ||
| - linux/s390x | ||
| - linux-m2xlarge/arm64 | ||
| - name: image-expires-after | ||
| value: 5d | ||
| - name: dockerfile | ||
| value: Dockerfile.konflux.automl | ||
| - name: path-context | ||
| value: . | ||
| - name: hermetic | ||
| value: true | ||
| - name: prefetch-input | ||
| value: | ||
| [ | ||
| { | ||
| "type": "pip", | ||
| "path": "pipelines/training/automl", | ||
| "requirements_files": [ | ||
| "autogluon_tabular_training_pipeline/requirements.txt" | ||
| ], | ||
| "binary": {"arch": ":all:"} | ||
| } | ||
| ] | ||
| - name: build-image-index | ||
| value: true | ||
| - name: enable-slack-failure-notification | ||
| value: "false" | ||
| pipelineRef: | ||
| resolver: git | ||
| params: | ||
| - name: url | ||
| value: https://github.com/red-hat-data-services/konflux-central.git | ||
| - name: revision | ||
| value: '{{ target_branch }}' | ||
| - name: pathInRepo | ||
| value: pipelines/multi-arch-container-build.yaml | ||
| taskRunTemplate: | ||
| serviceAccountName: build-pipeline-pull-request-pipelines | ||
| workspaces: | ||
| - name: git-auth | ||
| secret: | ||
| secretName: '{{ git_auth_secret }}' | ||
| status: {} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,96 @@ | ||||||
| apiVersion: tekton.dev/v1 | ||||||
| kind: PipelineRun | ||||||
| metadata: | ||||||
| annotations: | ||||||
| build.appstudio.openshift.io/repo: https://github.com/red-hat-data-services/pipelines-components?rev={{revision}} | ||||||
| build.appstudio.redhat.com/commit_sha: '{{revision}}' | ||||||
| build.appstudio.redhat.com/target_branch: '{{target_branch}}' | ||||||
| build.appstudio.redhat.com/pull_request_number: "{{pull_request_number}}" | ||||||
| pipelinesascode.tekton.dev/max-keep-runs: "3" | ||||||
| pipelinesascode.tekton.dev/on-comment: "^/build-konflux-autorag" | ||||||
| pipelinesascode.tekton.dev/on-event: "[pull_request]" | ||||||
| pipelinesascode.tekton.dev/cancel-in-progress: "true" | ||||||
| labels: | ||||||
| appstudio.openshift.io/application: automation | ||||||
| appstudio.openshift.io/component: pull-request-pipelines-odh-autorag | ||||||
| pipelines.appstudio.openshift.io/type: build | ||||||
| name: odh-autorag-on-pull-request | ||||||
| namespace: rhoai-tenant | ||||||
| spec: | ||||||
| timeouts: | ||||||
| pipeline: 8h | ||||||
| params: | ||||||
| - name: git-url | ||||||
| value: '{{source_url}}' | ||||||
| - name: revision | ||||||
| value: '{{revision}}' | ||||||
| - name: additional-tags | ||||||
| value: | ||||||
| - 'pr-{{pull_request_number}}-into-{{target_branch}}' | ||||||
| - name: additional-labels | ||||||
| value: | ||||||
| - version=on-pr-{{revision}} | ||||||
| - io.openshift.tags=odh-autorag | ||||||
| - name: output-image | ||||||
| value: quay.io/rhoai/pull-request-pipelines:odh-autorag-{{revision}} | ||||||
| - name: build-platforms | ||||||
| value: | ||||||
| - linux/x86_64 | ||||||
| - linux/ppc64le | ||||||
| - linux/s390x | ||||||
| - linux-m2xlarge/arm64 | ||||||
| - name: image-expires-after | ||||||
| value: 5d | ||||||
| - name: dockerfile | ||||||
| value: Dockerfile.konflux.autorag | ||||||
| - name: path-context | ||||||
| value: . | ||||||
| - name: hermetic | ||||||
| value: true | ||||||
| - name: prefetch-input | ||||||
| value: | ||||||
| [ | ||||||
| { | ||||||
| "type": "pip", | ||||||
| "path": "pipelines/training/autorag", | ||||||
| "requirements_files": [ | ||||||
| "documents_rag_optimization_pipeline/requirements.txt", | ||||||
| "documents_rag_optimization_pipeline/requirements-pypi-whl.txt" | ||||||
| ], | ||||||
| "binary": {"arch": ":all:"} | ||||||
| }, | ||||||
| { | ||||||
| "type": "pip", | ||||||
| "path": "pipelines/training/autorag/documents_rag_optimization_pipeline", | ||||||
| "requirements_files": [ | ||||||
| "requirements-pypi.txt" | ||||||
| ] | ||||||
| }, | ||||||
| { | ||||||
| "type": "generic", | ||||||
| "path": "pipelines/training/autorag/documents_rag_optimization_pipeline", | ||||||
| "lockfile": "artifacts.lock.yaml" | ||||||
| } | ||||||
| ] | ||||||
| - name: prefetch-log-level | ||||||
| value: "debug" | ||||||
|
Comment on lines
+75
to
+76
There was a problem hiding this comment. Debug logging may leak sensitive information. Setting Remove or set to 🔒 Proposed fix- - name: prefetch-log-level
- value: "debug"📝 Committable suggestion
Suggested change
🤖 Prompt for AI Agents |
||||||
| - name: build-image-index | ||||||
| value: true | ||||||
| - name: enable-slack-failure-notification | ||||||
| value: "false" | ||||||
| pipelineRef: | ||||||
| resolver: git | ||||||
| params: | ||||||
| - name: url | ||||||
| value: https://github.com/red-hat-data-services/konflux-central.git | ||||||
| - name: revision | ||||||
| value: '{{ target_branch }}' | ||||||
| - name: pathInRepo | ||||||
| value: pipelines/multi-arch-container-build.yaml | ||||||
| taskRunTemplate: | ||||||
| serviceAccountName: build-pipeline-pull-request-pipelines | ||||||
| workspaces: | ||||||
| - name: git-auth | ||||||
| secret: | ||||||
| secretName: '{{ git_auth_secret }}' | ||||||
| status: {} | ||||||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| --- | ||
| apiVersion: tekton.dev/v1 | ||
| kind: PipelineRun | ||
| metadata: | ||
| annotations: | ||
| build.appstudio.openshift.io/repo: https://github.com/red-hat-data-services/pipelines-components?rev={{revision}} | ||
| build.appstudio.redhat.com/commit_sha: '{{revision}}' | ||
| build.appstudio.redhat.com/target_branch: '{{target_branch}}' | ||
| build.appstudio.redhat.com/pull_request_number: "{{pull_request_number}}" | ||
| pipelinesascode.tekton.dev/max-keep-runs: "3" | ||
| pipelinesascode.tekton.dev/on-comment: "^/build-konflux odh-pipelines-components" | ||
| pipelinesascode.tekton.dev/cancel-in-progress: "true" | ||
| labels: | ||
| appstudio.openshift.io/application: automation | ||
| appstudio.openshift.io/component: pull-request-pipelines-odh-pipelines-components | ||
| pipelines.appstudio.openshift.io/type: build | ||
| name: odh-pipelines-components-on-pull-request-{{pull_request_number}} | ||
| namespace: rhoai-tenant | ||
| spec: | ||
| params: | ||
| - name: git-url | ||
| value: '{{source_url}}' | ||
| - name: revision | ||
| value: '{{revision}}' | ||
| - name: additional-tags | ||
| value: | ||
| - 'pr-{{pull_request_number}}-into-{{target_branch}}' | ||
| - name: additional-labels | ||
| value: | ||
| - version=on-pr-{{revision}} | ||
| - io.openshift.tags=odh-pipelines-components | ||
| - name: output-image | ||
| value: quay.io/rhoai/pull-request-pipelines:odh-pipelines-components-{{revision}} | ||
| - name: dockerfile | ||
| value: Dockerfile.konflux.pipelines-components | ||
| - name: path-context | ||
| value: . | ||
| - name: hermetic | ||
| value: 'true' | ||
| - name: prefetch-input | ||
| value: >- | ||
| {"type": "pip", "path": ".", | ||
| "requirements_files": ["requirements.txt"], | ||
| "requirements_build_files": ["requirements-build.txt"], | ||
| "binary": {"arch": ":all:"}} | ||
|
|
||
| - name: build-source-image | ||
| value: true | ||
| - name: build-image-index | ||
| value: true | ||
| - name: build-platforms | ||
| value: | ||
| - linux/x86_64 | ||
| - linux-m2xlarge/arm64 | ||
| - linux/ppc64le | ||
| - linux/s390x | ||
| - name: image-expires-after | ||
| value: 5d | ||
| - name: enable-slack-failure-notification | ||
| value: "false" | ||
| pipelineRef: | ||
| resolver: git | ||
| params: | ||
| - name: url | ||
| value: https://github.com/red-hat-data-services/konflux-central.git | ||
| - name: revision | ||
| value: '{{ target_branch }}' | ||
| - name: pathInRepo | ||
| value: pipelines/multi-arch-container-build.yaml | ||
| taskRunTemplate: | ||
| serviceAccountName: build-pipeline-pull-request-pipelines | ||
| workspaces: | ||
| - name: git-auth | ||
| secret: | ||
| secretName: '{{git_auth_secret}}' | ||
| status: {} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| ARG BASE_IMAGE="registry.redhat.io/rhai/base-image-cpu-rhel9@sha256:e42e1d0b52b4f2d2906b302f75987d353248279fa69eead69f907f6fff8d708c" | ||
|
|
||
| FROM ${BASE_IMAGE} | ||
|
|
||
| COPY pipelines/training/automl/autogluon_tabular_training_pipeline/requirements.txt \ | ||
| pipelines/training/automl/autogluon_tabular_training_pipeline/requirements-pypi.txt \ | ||
| /tmp/ | ||
|
|
||
| ## Ensure build isolation | ||
| RUN pip install --index-url https://console.redhat.com/api/pypi/public-rhai/rhoai/3.4/cpu-ubi9/simple/ --no-cache-dir -r /tmp/requirements.txt | ||
| RUN pip install --index-url https://pypi.org/simple --no-cache-dir -r /tmp/requirements-pypi.txt | ||
|
Comment on lines
+10
to
+11
There was a problem hiding this comment. Potential dependency conflict from sequential index installs. Installing from two different package indexes sequentially (RHAI then PyPI) can cause dependency resolution conflicts if the same package exists in both indexes with different versions. The second install may override packages from the first, breaking the "build isolation" mentioned in line 9. Consider consolidating requirements or using 🤖 Prompt for AI Agents |
||
|
|
||
| LABEL com.redhat.component="odh-automl-container" \ | ||
| name="managed-open-data-hub/odh-automl-rhel9" \ | ||
| description="odh-automl" \ | ||
| summary="odh-automl" \ | ||
| maintainer="['managed-open-data-hub@redhat.com']" \ | ||
| io.openshift.expose-services="" \ | ||
| io.k8s.display-name="odh-automl" \ | ||
| io.k8s.description="odh-automl" \ | ||
| com.redhat.license_terms="https://www.redhat.com/licenses/Red_Hat_Standard_EULA_20191108.pdf" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| # This container is used to run the AutoRAG components of the training pipeline. | ||
|
|
||
| #ARG BASE_IMAGE="quay.io/aipcc/base-images/cpu:3.3" | ||
| ARG BASE_IMAGE="registry.redhat.io/rhai/base-image-cpu-rhel9@sha256:e42e1d0b52b4f2d2906b302f75987d353248279fa69eead69f907f6fff8d708c" | ||
| FROM ${BASE_IMAGE} | ||
|
|
||
| COPY pipelines/training/autorag/documents_rag_optimization_pipeline/requirements.txt \ | ||
| pipelines/training/autorag/documents_rag_optimization_pipeline/requirements-pypi.txt \ | ||
| pipelines/training/autorag/documents_rag_optimization_pipeline/requirements-pypi-whl.txt \ | ||
| /tmp/ | ||
|
|
||
| COPY pipelines/training/autorag/documents_rag_optimization_pipeline/seed_docling_models.py \ | ||
| pipelines/training/autorag/documents_rag_optimization_pipeline/artifacts.lock.yaml \ | ||
| pipelines/training/autorag/documents_rag_optimization_pipeline/install_sqlite_from_source.sh \ | ||
| /tmp/ | ||
|
|
||
| # Build SQLite from sqlite-autoconf (Hermeto generic prefetch or download); replace system libsqlite3. | ||
| # https://github.com/hermetoproject/hermeto/blob/main/docs/generic.md | ||
| ARG HERMETO_GENERIC_DEPS=/cachi2/output/deps/generic | ||
|
|
||
| # Build SQLite from sqlite-autoconf (Hermeto generic prefetch or download); replace system libsqlite3. | ||
| USER root | ||
| RUN HERMETO_GENERIC_DEPS="${HERMETO_GENERIC_DEPS}" bash /tmp/install_sqlite_from_source.sh | ||
|
Comment on lines
+22
to
+23
There was a problem hiding this comment. Arbitrary script execution as root (CWE-250). Switching to Refactor to:
As per coding guidelines, Dockerfiles should run as non-root user. 🤖 Prompt for AI Agents |
||
| USER default | ||
|
|
||
| ## TODO: Ensure build isolation | ||
| RUN pip install --index-url https://console.redhat.com/api/pypi/public-rhai/rhoai/3.4/cpu-ubi9/simple/ --no-cache-dir -r /tmp/requirements.txt | ||
| RUN pip install --index-url https://pypi.org/simple --no-cache-dir -r /tmp/requirements-pypi.txt | ||
| RUN pip install --index-url https://pypi.org/simple --no-cache-dir -r /tmp/requirements-pypi-whl.txt | ||
|
Comment on lines
+26
to
+29
There was a problem hiding this comment. Three sequential pip installs risk dependency conflicts. Installing from three different package indexes sequentially (RHAI → PyPI → PyPI wheel) increases the risk of version conflicts and dependency resolution failures. The TODO comment on line 26 acknowledges that build isolation is not ensured. Consolidate into a single requirements file with explicit source annotations, or use a lock file (e.g., 🤖 Prompt for AI Agents |
||
|
|
||
| # Docling models for offline text extraction. Hermeto: prefetch with generic lockfile (see /tmp/artifacts.lock.yaml) | ||
| # and mount output at ${HERMETO_GENERIC_DEPS}; https://github.com/hermetoproject/hermeto/blob/main/docs/generic.md | ||
| ENV DOCLING_ARTIFACTS_PATH=/opt/app-root/docling-artifacts/models | ||
| RUN if [ -d "${HERMETO_GENERIC_DEPS}" ] && [ -n "$(ls -A "${HERMETO_GENERIC_DEPS}" 2>/dev/null)" ]; then \ | ||
| python3 /tmp/seed_docling_models.py --dest /opt/app-root/docling-artifacts/models --hermeto-dir "${HERMETO_GENERIC_DEPS}"; \ | ||
| else \ | ||
| python3 /tmp/seed_docling_models.py --dest /opt/app-root/docling-artifacts/models --download; \ | ||
| fi | ||
|
|
||
| LABEL com.redhat.component="odh-autorag-container" \ | ||
| name="managed-open-data-hub/odh-autorag-rhel9" \ | ||
| description="odh-autorag" \ | ||
| summary="odh-autorag" \ | ||
| maintainer="['managed-open-data-hub@redhat.com']" \ | ||
| io.openshift.expose-services="" \ | ||
| io.k8s.display-name="odh-autorag" \ | ||
| io.k8s.description="odh-autorag" \ | ||
| com.redhat.license_terms="https://www.redhat.com/licenses/Red_Hat_Standard_EULA_20191108.pdf" | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replace direct-to-main instructions with PR-based flow.
Current instructions tell contributors to push straight to
main, which weakens change control and can bypass required reviews/checks if branch protection is permissive. This is a supply-chain/process security risk (CWE-285: Improper Authorization at workflow level).Suggested doc fix
Add a final step instructing users to open a PR against
maininred-hat-data-services/konflux-central.As per coding guidelines, "REVIEW PRIORITIES: 1. Security vulnerabilities ... 2. Architectural issues and anti-patterns".
Also applies to: 35-37
🤖 Prompt for AI Agents