Skip to content

Enable pr testing#1

Closed
Fiona-Waters wants to merge 48 commits intomainfrom
enable-pr-testing
Closed

Enable pr testing#1
Fiona-Waters wants to merge 48 commits intomainfrom
enable-pr-testing

Conversation

@Fiona-Waters
Copy link
Copy Markdown
Owner

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, in Fixes #<issue number>, #<issue number>, ... format, will close the issue(s) when PR gets merged):

Fixes #

Checklist:

  • Docs included if any changes are user facing

andreyvelich and others added 30 commits October 8, 2025 15:02
…es() API (kubeflow#64)

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Implement Kubernetes Backends

Signed-off-by: Saad Zaher <szaher@redhat.com>

* add k8s_test to test kubernetes backend

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* run ruff format

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* address reviews

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* Enabling KubernetesBackendConfig to use arbitrary types

Kubernetes client configuration isn't supported as pydantic core
schema so we need to ask pydantic to allow arbitrary types.

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* Remove BaseBackend and use BaseModel

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Rename base backend to ExecutionBackend

Signed-off-by: Saad Zaher <szaher@redhat.com>

* fix api trainer_client order

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* move delete_job func to the end of trainer_client

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

---------

Signed-off-by: Saad Zaher <szaher@redhat.com>
Signed-off-by: Saad Zaher <eng.szaher@gmail.com>
* Initial update of README.md

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* update links in README

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* add persona diagram to README

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* update persona diagram link in REDAME

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Update naming in README.md

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Anya Kramar <akramar@redhat.com>

* remove license badge from README

Signed-off-by: kramaranya <kramaranya15@gmail.com>

---------

Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: Anya Kramar <akramar@redhat.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore: Move dev extras to dependency-groups

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* update uv sync command

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* add development build to CONTRIBUTING

Signed-off-by: kramaranya <kramaranya15@gmail.com>

---------

Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: Anya Kramar <kramaranya15@gmail.com>
Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: Akash Jaiswal <akashjaiswal3846@gmail.com>
Signed-off-by: kramaranya <kramaranya15@gmail.com>
…beflow#76)

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore(trainer): Use explicit exception chaining

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

* Update kubeflow/trainer/api/trainer_client.py

Co-authored-by: Anya Kramar <akramar@redhat.com>
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

* Use builtin types in all files

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

---------

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Co-authored-by: Anya Kramar <akramar@redhat.com>
* feat(trainer): Refactor get_job_logs() API

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

* Fix unit tests

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

* Remove unused func

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

* Update kubeflow/trainer/api/trainer_client.py

Co-authored-by: Anya Kramar <akramar@redhat.com>
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

* Fix print logs

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

* Rename TrainerClient to KubernetesBackend in tests

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

* Remove empty return from watch stream logs

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

---------

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Co-authored-by: Anya Kramar <akramar@redhat.com>
* feat: Support multiple pip index URLs in CustomTrainer

- Add pip_index_urls parameter to CustomTrainer (list of URLs)
- Update constants to support multiple URLs with environment variable
- Modify utils to handle multiple index URLs in pip installation
- Update tests to use new parameter
- Fix mutable default issue with dataclasses.field(default_factory=...)

Closes kubeflow#72

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* test: add comprehensive test cases for multiple pip index URLs

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* refactor: simplify constants for pip index URLs

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* fix: remove Optional from pip_index_urls docstring to match implementation

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* test: adjust unit test for get_custom_trainer

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* fix: add missing Dict import in utils.py

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* refactor: move test cases to utils_test.py and convert to parametrized format

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* refactor: update utils test to use exact string matching instead of contains/not_contains checks

- Changed TestCase expected_output from Dict[str, Any] to str
- Replaced contains/not_contains logic with direct string equality
- Updated all test cases to specify complete expected script output
- Tests now assert the entire script returned by get_script_for_python_packages

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* feat: add multiple pip index URLs support with comprehensive tests

- Add support for multiple pip index URLs in CustomTrainer
- Update constants to handle comma-separated environment variables
- Add integration test for multiple pip index URLs flow
- Update utils to generate correct pip install commands with --index-url and --extra-index-url
- Ensure backward compatibility with single URL usage

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* refactor: remove redundant multiple pip index URLs test from backend

- Remove redundant test case since comprehensive tests already exist in utils test
- Keep integration tests focused on core functionality rather than implementation details
- Maintain test coverage while reducing duplication

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* Revert "refactor: remove redundant multiple pip index URLs test from backend"

This reverts commit 0ec003d.

Signed-off-by: wassimbensalem <bswassim@gmail.com>

* refactor: remove duplicated test case from the backend.

Signed-off-by: wassimbensalem <bswassim@gmail.com>

---------

Signed-off-by: wassimbensalem <bswassim@gmail.com>
…on call

Signed-off-by: Brian Gallagher <briangal@gmail.com>
* chore: Add welcome new contributors CI

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Add community links to welcome CI

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Add slack channels to CI

Signed-off-by: kramaranya <kramaranya15@gmail.com>

---------

Signed-off-by: kramaranya <kramaranya15@gmail.com>
* KEP-2: Local Execution Mode Proposal

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Updated proposal

    * add apple containers
    * fix typo in Subprocess
    * add API consistency to the design details

Signed-off-by: Saad Zaher <szaher@redhat.com>

* update proposal to use training backends

Signed-off-by: Saad Zaher <szaher@redhat.com>

* add constraint on resource limitation for local mode

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Move proposals into docs

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Use ExecutionBackends instead of TrainingBackends

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* update docs and graphs

Signed-off-by: Saad Zaher <szaher@redhat.com>

* update graphs

Signed-off-by: Saad Zaher <szaher@redhat.com>

---------

Signed-off-by: Saad Zaher <szaher@redhat.com>
Signed-off-by: Saad Zaher <eng.szaher@gmail.com>
* Update CONTRIBUTING.md to use uv

Signed-off-by: Saad Zaher <szaher@redhat.com>

* remove .flake8 & update .gitignore

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Fix merge conflict

Signed-off-by: Saad Zaher <szaher@redhat.com>

* checkout Makefile, pyproject, lock file from main branch

Signed-off-by: Saad Zaher <szaher@redhat.com>

* update gitignore

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Update CONTRIBUTING.md

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>

* Add make install & foramt

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* Remove uv pip install dev dependencies

as we are moving to `dependency-groups` instead of `optional-dependencies` because pypi.org doesn't support git dependencies we need ot use `uv sync` instead of `uv pip ...` 
which will automatically sync all default groups (dev is a default group in uv)


Signed-off-by: Saad Zaher <szaher@redhat.com>

* fix typo in the contributing.md file

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Update Makefile

Signed-off-by: Saad Zaher <szaher@redhat.com>

* update structure

Signed-off-by: Saad Zaher <szaher@redhat.com>

* rephrase uv install

Signed-off-by: Saad Zaher <szaher@redhat.com>

* update install step at Makefile

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Update CONTRIBUTING.md

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>

* Apply suggestion from @andreyvelich

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>

* Apply suggestion from @andreyvelich

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>

* update Make target verify to use install

Signed-off-by: Saad Zaher <szaher@redhat.com>

* fix formatting in contributing guide

Signed-off-by: Saad Zaher <szaher@redhat.com>

* remove redundant commands

Signed-off-by: Saad Zaher <szaher@redhat.com>

* use make install-dev instead of make install

Signed-off-by: Saad Zaher <szaher@redhat.com>

* update install target reference

Signed-off-by: Saad Zaher <szaher@redhat.com>

* remove group dev from make install-dev target

Signed-off-by: Saad Zaher <szaher@redhat.com>

* move coding style to dev

Signed-off-by: Saad Zaher <szaher@redhat.com>

* remove pre-commit stuff from contributing guide

Signed-off-by: Saad Zaher <szaher@redhat.com>

---------

Signed-off-by: Saad Zaher <szaher@redhat.com>
Signed-off-by: Saad Zaher <eng.szaher@gmail.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Implement TrainerClient Backends & Local Process

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Implement Job Cancellation

Signed-off-by: Saad Zaher <szaher@redhat.com>

* update local job to add resouce limitation in k8s style

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Update python/kubeflow/trainer/api/trainer_client.py

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>

* Fix linting issues

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* fix unit tests

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* add support wait_for_job_status

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* Update data types

Signed-off-by: Saad Zaher <szaher@redhat.com>

* fix merge conflict

Signed-off-by: Saad Zaher <szaher@redhat.com>

* fix unit tests

Signed-off-by: Saad Zaher <szaher@redhat.com>

* remove TypeAlias

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Replace TRAINER_BACKEND_REGISTRY with TRAINER_BACKEND

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Update kubeflow/trainer/api/trainer_client.py

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>

* Update kubeflow/trainer/api/trainer_client.py

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>

* Restructure training backends into separate dirs

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Update kubeflow/trainer/api/trainer_client.py

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>

* add get_runtime_packages as not supported by local-exec

Signed-off-by: Saad Zaher <szaher@redhat.com>

* move backends and its configs to kubeflow.trainer

Signed-off-by: Saad Zaher <szaher@redhat.com>

* fix typo in delete_job

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Move local_runtimes to constants

 * Move local_runtimes to constants
 * allow list_jobs to filter by runtime
 * keep runtime ref in __local_jobs

Signed-off-by: Saad Zaher <szaher@redhat.com>

* use google style docstring for LocalJob

Signed-off-by: Saad Zaher <szaher@redhat.com>

* remove debug opt from LocalProcessConfig

Signed-off-by: Saad Zaher <szaher@redhat.com>

* only use imports from kubeflow.trainer for backends

Signed-off-by: Saad Zaher <szaher@redhat.com>

* upload local-exec to use only one step

While I believe in simplicity and diving this into steps makes it easier
for debugging and extensibility. Addressing comments on this PR
consolidating all train job scripts into one and running it as single
step to match k8s.

Signed-off-by: Saad Zaher <szaher@redhat.com>

* optimize loops when getting runtime

Signed-off-by: Saad Zaher <szaher@redhat.com>

* add LocalRuntimeTrainer

Signed-off-by: Saad Zaher <szaher@redhat.com>

* rename cleanup config item to cleanup_venv

Signed-off-by: Saad Zaher <szaher@redhat.com>

* convert local runtime to runtime

Signed-off-by: Saad Zaher <szaher@redhat.com>

* convert runtimes before returning

Signed-off-by: Saad Zaher <szaher@redhat.com>

* fix get_job_logs to align with parent interface

Signed-off-by: Saad Zaher <szaher@redhat.com>

* rename get_runtime_trainer func

Signed-off-by: Saad Zaher <szaher@redhat.com>

* rename get_training_job_command to get_local_train_job_script

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Ignore failures in Coveralls action

Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>

---------

Signed-off-by: Saad Zaher <szaher@redhat.com>
Signed-off-by: Saad Zaher <eng.szaher@gmail.com>
Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore: Add proper ruff configuration

Add proper ruff configuration to
* support proper linting
* support ruff format
* support isort with ruff
* update pre-commit configs

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>

* merge with main

Signed-off-by: Saad Zaher <szaher@redhat.com>

* fix format

Signed-off-by: Saad Zaher <szaher@redhat.com>

* fix linting issues

Signed-off-by: Saad Zaher <szaher@redhat.com>

* Update Makefile

remove reduandant `uv sync`

Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>

* let linter handle line-length \n

add labels next to linters codes

Signed-off-by: Saad Zaher <szaher@redhat.com>

* remove duplicate labels from linter

Signed-off-by: Saad Zaher <szaher@redhat.com>

* remove unwanted comment in pyproject.toml

Signed-off-by: Saad Zaher <szaher@redhat.com>

---------

Signed-off-by: Saad Zaher <eng.szaher@gmail.com>
Signed-off-by: Saad Zaher <szaher@redhat.com>
Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* fix(docs): Update links before SDK release

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* remove python versions badge

Signed-off-by: kramaranya <kramaranya15@gmail.com>

---------

Signed-off-by: kramaranya <kramaranya15@gmail.com>
* feat(ci): Add release CI job

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* feat(ci): Add GitHub Release to CI

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Change kubeflow sdk PyPI package name

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Add kubeflow sdk version verification to CI

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Add proper changelog extraction

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Allow to reuse test-python workflow

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Add changelog generation script

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Update uv.lock

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Remove blank lines in changelog script

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Add make release in Makefile

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Refactor gen_changelog

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Delete CHANGELOG.md

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Add prepare-release CI

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Allow manual trigger of release CI

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Add release readme

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Update chnagelog parser in CI

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* skip and check cherry pick for new release

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* add script to update the version

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Update make release script

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Use make test-python in release CI

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Move prepare release CI to one a single workflow

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Remove uv.sync from RELEASE.md

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Upload artifacts to GitHub Release

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Run release CI against release-* branch

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Use X.Y.Z versioning in Kubeflow SDK

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Use single workflow for release

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Remove Release types from RELEASE.md

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Add a note about older minor series patch release

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Directly checkout release branch with ref input

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Checkout release branch with ref input

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Update extract changelog job

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Update the name of github release

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Remove full changelog line

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Fix ruff issue

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Don't use link for authors in changelog

Signed-off-by: kramaranya <kramaranya15@gmail.com>

* Remove chnagelog for RCs

Signed-off-by: kramaranya <kramaranya15@gmail.com>

---------

Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: Brian Gallagher <briangal@gmail.com>
Signed-off-by: Brian Gallagher <briangal@gmail.com>
@coveralls
Copy link
Copy Markdown

coveralls commented Nov 11, 2025

Pull Request Test Coverage Report for Build 19370620691

Details

  • 1 of 1 (100.0%) changed or added relevant line in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 66.667%

Totals Coverage Status
Change from base Build 19142912008: 0.0%
Covered Lines: 2506
Relevant Lines: 3759

💛 - Coveralls

@Fiona-Waters Fiona-Waters force-pushed the enable-pr-testing branch 14 times, most recently from 36fbdc3 to 716dea7 Compare November 14, 2025 16:09
@github-actions
Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants