ITEP-64780 weights uploader

[mgumowsk]

📝 Description

In order to overcome the problem with installation timeouts due to prolonged initial weights download, we changed the way they are copied to Geti.
In details:

• Instead of having all weights inside of an image, we keep there only the pretrained_models_v2.json file. This file should be sent to S3 as the first one based on a content of this file it should download the file with weights from the location given in the WEIGHTS_URL parameter passed to the pod. Each file should be then sent to S3 - as it is done now.
• If certain model already exists in S3, the algorithm skips it and moves to another one, adding information about this fact to logs.
• The weights download job waits until seaweedfs is been installed, but it does not block the installer.
• WEIGHTS_URL env is passed to the job to indicate the location of the weights. The default value set to the https://storage.geti.infra-host.com/weights value. Additionally, this job isn't executed if the DISABLE_WEIGHT_UPLOADING parameter is set to true.
• As irrelevant anymore, local_mirror has been removed from pretrained_models_v2.json.

Fixes #147

✨ Type of Change

Select the type of change your PR introduces:

• 🐞 Bug fix – Non-breaking change which fixes an issue
• 🚀 New feature – Non-breaking change which adds functionality
• 🔨 Refactor – Non-breaking change which refactors the code base
• 💥 Breaking change – Changes that break existing functionality
• 📚 Documentation update
• 🔒 Security update
• 🧪 Tests

🧪 Testing Scenarios

• ✅ Tested manually
• 🤖 Run automated end-to-end tests

✅ Checklist

Before submitting the PR, ensure the following:

• 🔍 PR title is clear and descriptive
• 📝 For internal contributors: If applicable, include the JIRA ticket number (e.g., ITEP-123456) in the PR title. Do not include full URLs
• 💬 I have commented my code, especially in hard-to-understand areas
• 📄 I have made corresponding changes to the documentation
• ✅ I have added tests that prove my fix is effective or my feature works

[Copilot]

Pull Request Overview

This PR refactors and implements the weights uploader service (ITEP-64780), adding configuration, unit tests, Helm charts, and updating related helper scripts.

• Added unit tests for main in test_weights_uploader.py
• Defined project metadata and dependencies in pyproject.toml
• Introduced Helm chart templates for ServiceEntry, RBAC, proxy, and Job resources
• Implemented core upload/download logic in app/weights_uploader.py and app/pretrained_models/pretrained_models.py
• Updated interactive AI workflows to use the new download utilities and exposed WEIGHTS_URL in pod definitions

Reviewed Changes

Copilot reviewed 15 out of 19 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
platform/services/weights_uploader/tests/unit/test_weights_uploader.py	Added auto-mocked env var fixture and success/failure tests
platform/services/weights_uploader/pyproject.toml	Project metadata, dependencies, and tooling config
platform/services/weights_uploader/chart/templates/*.yaml	Added Helm templates for ServiceEntry, RBAC, proxy, and Job
platform/services/weights_uploader/app/weights_uploader.py	Core S3 upload logic and main workflow
platform/services/weights_uploader/app/pretrained_models/pretrained_models.py	Download, unzip, retry, and SHA‐256 verification logic
interactive_ai/.../scripts/run.py	Switched calls from download_file_from_s3 to download_file
interactive_ai/.../scripts/minio_util.py	Refactored download utilities, added URL fallback
interactive_ai/.../trainer_pod_definition.py	Exposed WEIGHTS_URL in Kubernetes container environment

Files not reviewed (4)

• platform/services/weights_uploader/.dockerignore: Language not supported
• platform/services/weights_uploader/Dockerfile: Language not supported
• platform/services/weights_uploader/Makefile: Language not supported
• platform/services/weights_uploader/app/pretrained_models/pretrained_models_v2.json: Language not supported

Comments suppressed due to low confidence (3)

platform/services/weights_uploader/pyproject.toml:6

• There's a typo in the description—"Wights" should be "Weights".

description = "Wights uploader service for Geti"

interactive_ai/workflows/otx_domain/trainer/otx_v2/scripts/minio_util.py:16

• The download_file_from_url function uses bucket_name inside but does not accept it as a parameter, causing an undefined variable at runtime.

def download_file_from_url(client: Minio, object_name: str, file_path: str) -> None:

interactive_ai/workflows/otx_domain/trainer/otx_v2/scripts/minio_util.py:23

• requests is used here but not imported at the top of the module, leading to a NameError.

resp = requests.get(url, timeout=600)

[AlexanderBarabanov]

Thanks!