Closed
Conversation
…es() API (kubeflow#64) Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Implement Kubernetes Backends Signed-off-by: Saad Zaher <szaher@redhat.com> * add k8s_test to test kubernetes backend Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * run ruff format Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * address reviews Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * Enabling KubernetesBackendConfig to use arbitrary types Kubernetes client configuration isn't supported as pydantic core schema so we need to ask pydantic to allow arbitrary types. Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * Remove BaseBackend and use BaseModel Signed-off-by: Saad Zaher <szaher@redhat.com> * Rename base backend to ExecutionBackend Signed-off-by: Saad Zaher <szaher@redhat.com> * fix api trainer_client order Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * move delete_job func to the end of trainer_client Signed-off-by: Saad Zaher <eng.szaher@gmail.com> --------- Signed-off-by: Saad Zaher <szaher@redhat.com> Signed-off-by: Saad Zaher <eng.szaher@gmail.com>
* Initial update of README.md Signed-off-by: kramaranya <kramaranya15@gmail.com> * update links in README Signed-off-by: kramaranya <kramaranya15@gmail.com> * add persona diagram to README Signed-off-by: kramaranya <kramaranya15@gmail.com> * update persona diagram link in REDAME Signed-off-by: kramaranya <kramaranya15@gmail.com> * Update naming in README.md Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Anya Kramar <akramar@redhat.com> * remove license badge from README Signed-off-by: kramaranya <kramaranya15@gmail.com> --------- Signed-off-by: kramaranya <kramaranya15@gmail.com> Signed-off-by: Anya Kramar <akramar@redhat.com> Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore: Move dev extras to dependency-groups Signed-off-by: kramaranya <kramaranya15@gmail.com> * update uv sync command Signed-off-by: kramaranya <kramaranya15@gmail.com> * add development build to CONTRIBUTING Signed-off-by: kramaranya <kramaranya15@gmail.com> --------- Signed-off-by: kramaranya <kramaranya15@gmail.com> Signed-off-by: Anya Kramar <kramaranya15@gmail.com>
Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: Akash Jaiswal <akashjaiswal3846@gmail.com>
Signed-off-by: kramaranya <kramaranya15@gmail.com>
…beflow#76) Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore(trainer): Use explicit exception chaining Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Update kubeflow/trainer/api/trainer_client.py Co-authored-by: Anya Kramar <akramar@redhat.com> Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Use builtin types in all files Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> --------- Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Co-authored-by: Anya Kramar <akramar@redhat.com>
* feat(trainer): Refactor get_job_logs() API Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Fix unit tests Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Remove unused func Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Update kubeflow/trainer/api/trainer_client.py Co-authored-by: Anya Kramar <akramar@redhat.com> Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Fix print logs Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Rename TrainerClient to KubernetesBackend in tests Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> * Remove empty return from watch stream logs Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> --------- Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Co-authored-by: Anya Kramar <akramar@redhat.com>
* feat: Support multiple pip index URLs in CustomTrainer - Add pip_index_urls parameter to CustomTrainer (list of URLs) - Update constants to support multiple URLs with environment variable - Modify utils to handle multiple index URLs in pip installation - Update tests to use new parameter - Fix mutable default issue with dataclasses.field(default_factory=...) Closes kubeflow#72 Signed-off-by: wassimbensalem <bswassim@gmail.com> * test: add comprehensive test cases for multiple pip index URLs Signed-off-by: wassimbensalem <bswassim@gmail.com> * refactor: simplify constants for pip index URLs Signed-off-by: wassimbensalem <bswassim@gmail.com> * fix: remove Optional from pip_index_urls docstring to match implementation Signed-off-by: wassimbensalem <bswassim@gmail.com> * test: adjust unit test for get_custom_trainer Signed-off-by: wassimbensalem <bswassim@gmail.com> * fix: add missing Dict import in utils.py Signed-off-by: wassimbensalem <bswassim@gmail.com> * refactor: move test cases to utils_test.py and convert to parametrized format Signed-off-by: wassimbensalem <bswassim@gmail.com> * refactor: update utils test to use exact string matching instead of contains/not_contains checks - Changed TestCase expected_output from Dict[str, Any] to str - Replaced contains/not_contains logic with direct string equality - Updated all test cases to specify complete expected script output - Tests now assert the entire script returned by get_script_for_python_packages Signed-off-by: wassimbensalem <bswassim@gmail.com> * feat: add multiple pip index URLs support with comprehensive tests - Add support for multiple pip index URLs in CustomTrainer - Update constants to handle comma-separated environment variables - Add integration test for multiple pip index URLs flow - Update utils to generate correct pip install commands with --index-url and --extra-index-url - Ensure backward compatibility with single URL usage Signed-off-by: wassimbensalem <bswassim@gmail.com> * refactor: remove redundant multiple pip index URLs test from backend - Remove redundant test case since comprehensive tests already exist in utils test - Keep integration tests focused on core functionality rather than implementation details - Maintain test coverage while reducing duplication Signed-off-by: wassimbensalem <bswassim@gmail.com> * Revert "refactor: remove redundant multiple pip index URLs test from backend" This reverts commit 0ec003d. Signed-off-by: wassimbensalem <bswassim@gmail.com> * refactor: remove duplicated test case from the backend. Signed-off-by: wassimbensalem <bswassim@gmail.com> --------- Signed-off-by: wassimbensalem <bswassim@gmail.com>
…on call Signed-off-by: Brian Gallagher <briangal@gmail.com>
* chore: Add welcome new contributors CI Signed-off-by: kramaranya <kramaranya15@gmail.com> * Add community links to welcome CI Signed-off-by: kramaranya <kramaranya15@gmail.com> * Add slack channels to CI Signed-off-by: kramaranya <kramaranya15@gmail.com> --------- Signed-off-by: kramaranya <kramaranya15@gmail.com>
* KEP-2: Local Execution Mode Proposal
Signed-off-by: Saad Zaher <szaher@redhat.com>
* Updated proposal
* add apple containers
* fix typo in Subprocess
* add API consistency to the design details
Signed-off-by: Saad Zaher <szaher@redhat.com>
* update proposal to use training backends
Signed-off-by: Saad Zaher <szaher@redhat.com>
* add constraint on resource limitation for local mode
Signed-off-by: Saad Zaher <szaher@redhat.com>
* Move proposals into docs
Signed-off-by: Saad Zaher <szaher@redhat.com>
* Use ExecutionBackends instead of TrainingBackends
Signed-off-by: Saad Zaher <eng.szaher@gmail.com>
* update docs and graphs
Signed-off-by: Saad Zaher <szaher@redhat.com>
* update graphs
Signed-off-by: Saad Zaher <szaher@redhat.com>
---------
Signed-off-by: Saad Zaher <szaher@redhat.com>
Signed-off-by: Saad Zaher <eng.szaher@gmail.com>
* Update CONTRIBUTING.md to use uv Signed-off-by: Saad Zaher <szaher@redhat.com> * remove .flake8 & update .gitignore Signed-off-by: Saad Zaher <szaher@redhat.com> * Fix merge conflict Signed-off-by: Saad Zaher <szaher@redhat.com> * checkout Makefile, pyproject, lock file from main branch Signed-off-by: Saad Zaher <szaher@redhat.com> * update gitignore Signed-off-by: Saad Zaher <szaher@redhat.com> * Update CONTRIBUTING.md Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> * Add make install & foramt Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * Remove uv pip install dev dependencies as we are moving to `dependency-groups` instead of `optional-dependencies` because pypi.org doesn't support git dependencies we need ot use `uv sync` instead of `uv pip ...` which will automatically sync all default groups (dev is a default group in uv) Signed-off-by: Saad Zaher <szaher@redhat.com> * fix typo in the contributing.md file Signed-off-by: Saad Zaher <szaher@redhat.com> * Update Makefile Signed-off-by: Saad Zaher <szaher@redhat.com> * update structure Signed-off-by: Saad Zaher <szaher@redhat.com> * rephrase uv install Signed-off-by: Saad Zaher <szaher@redhat.com> * update install step at Makefile Signed-off-by: Saad Zaher <szaher@redhat.com> * Update CONTRIBUTING.md Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> * Apply suggestion from @andreyvelich Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> * Apply suggestion from @andreyvelich Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> * update Make target verify to use install Signed-off-by: Saad Zaher <szaher@redhat.com> * fix formatting in contributing guide Signed-off-by: Saad Zaher <szaher@redhat.com> * remove redundant commands Signed-off-by: Saad Zaher <szaher@redhat.com> * use make install-dev instead of make install Signed-off-by: Saad Zaher <szaher@redhat.com> * update install target reference Signed-off-by: Saad Zaher <szaher@redhat.com> * remove group dev from make install-dev target Signed-off-by: Saad Zaher <szaher@redhat.com> * move coding style to dev Signed-off-by: Saad Zaher <szaher@redhat.com> * remove pre-commit stuff from contributing guide Signed-off-by: Saad Zaher <szaher@redhat.com> --------- Signed-off-by: Saad Zaher <szaher@redhat.com> Signed-off-by: Saad Zaher <eng.szaher@gmail.com> Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* Implement TrainerClient Backends & Local Process Signed-off-by: Saad Zaher <szaher@redhat.com> * Implement Job Cancellation Signed-off-by: Saad Zaher <szaher@redhat.com> * update local job to add resouce limitation in k8s style Signed-off-by: Saad Zaher <szaher@redhat.com> * Update python/kubeflow/trainer/api/trainer_client.py Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> * Fix linting issues Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * fix unit tests Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * add support wait_for_job_status Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * Update data types Signed-off-by: Saad Zaher <szaher@redhat.com> * fix merge conflict Signed-off-by: Saad Zaher <szaher@redhat.com> * fix unit tests Signed-off-by: Saad Zaher <szaher@redhat.com> * remove TypeAlias Signed-off-by: Saad Zaher <szaher@redhat.com> * Replace TRAINER_BACKEND_REGISTRY with TRAINER_BACKEND Signed-off-by: Saad Zaher <szaher@redhat.com> * Update kubeflow/trainer/api/trainer_client.py Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> * Update kubeflow/trainer/api/trainer_client.py Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> * Restructure training backends into separate dirs Signed-off-by: Saad Zaher <szaher@redhat.com> * Update kubeflow/trainer/api/trainer_client.py Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> * add get_runtime_packages as not supported by local-exec Signed-off-by: Saad Zaher <szaher@redhat.com> * move backends and its configs to kubeflow.trainer Signed-off-by: Saad Zaher <szaher@redhat.com> * fix typo in delete_job Signed-off-by: Saad Zaher <szaher@redhat.com> * Move local_runtimes to constants * Move local_runtimes to constants * allow list_jobs to filter by runtime * keep runtime ref in __local_jobs Signed-off-by: Saad Zaher <szaher@redhat.com> * use google style docstring for LocalJob Signed-off-by: Saad Zaher <szaher@redhat.com> * remove debug opt from LocalProcessConfig Signed-off-by: Saad Zaher <szaher@redhat.com> * only use imports from kubeflow.trainer for backends Signed-off-by: Saad Zaher <szaher@redhat.com> * upload local-exec to use only one step While I believe in simplicity and diving this into steps makes it easier for debugging and extensibility. Addressing comments on this PR consolidating all train job scripts into one and running it as single step to match k8s. Signed-off-by: Saad Zaher <szaher@redhat.com> * optimize loops when getting runtime Signed-off-by: Saad Zaher <szaher@redhat.com> * add LocalRuntimeTrainer Signed-off-by: Saad Zaher <szaher@redhat.com> * rename cleanup config item to cleanup_venv Signed-off-by: Saad Zaher <szaher@redhat.com> * convert local runtime to runtime Signed-off-by: Saad Zaher <szaher@redhat.com> * convert runtimes before returning Signed-off-by: Saad Zaher <szaher@redhat.com> * fix get_job_logs to align with parent interface Signed-off-by: Saad Zaher <szaher@redhat.com> * rename get_runtime_trainer func Signed-off-by: Saad Zaher <szaher@redhat.com> * rename get_training_job_command to get_local_train_job_script Signed-off-by: Saad Zaher <szaher@redhat.com> * Ignore failures in Coveralls action Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> --------- Signed-off-by: Saad Zaher <szaher@redhat.com> Signed-off-by: Saad Zaher <eng.szaher@gmail.com> Signed-off-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* chore: Add proper ruff configuration Add proper ruff configuration to * support proper linting * support ruff format * support isort with ruff * update pre-commit configs Signed-off-by: Saad Zaher <eng.szaher@gmail.com> * merge with main Signed-off-by: Saad Zaher <szaher@redhat.com> * fix format Signed-off-by: Saad Zaher <szaher@redhat.com> * fix linting issues Signed-off-by: Saad Zaher <szaher@redhat.com> * Update Makefile remove reduandant `uv sync` Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> * let linter handle line-length \n add labels next to linters codes Signed-off-by: Saad Zaher <szaher@redhat.com> * remove duplicate labels from linter Signed-off-by: Saad Zaher <szaher@redhat.com> * remove unwanted comment in pyproject.toml Signed-off-by: Saad Zaher <szaher@redhat.com> --------- Signed-off-by: Saad Zaher <eng.szaher@gmail.com> Signed-off-by: Saad Zaher <szaher@redhat.com> Co-authored-by: Andrey Velichkevich <andrey.velichkevich@gmail.com>
* fix(docs): Update links before SDK release Signed-off-by: kramaranya <kramaranya15@gmail.com> * remove python versions badge Signed-off-by: kramaranya <kramaranya15@gmail.com> --------- Signed-off-by: kramaranya <kramaranya15@gmail.com>
* feat(ci): Add release CI job Signed-off-by: kramaranya <kramaranya15@gmail.com> * feat(ci): Add GitHub Release to CI Signed-off-by: kramaranya <kramaranya15@gmail.com> * Change kubeflow sdk PyPI package name Signed-off-by: kramaranya <kramaranya15@gmail.com> * Add kubeflow sdk version verification to CI Signed-off-by: kramaranya <kramaranya15@gmail.com> * Add proper changelog extraction Signed-off-by: kramaranya <kramaranya15@gmail.com> * Allow to reuse test-python workflow Signed-off-by: kramaranya <kramaranya15@gmail.com> * Add changelog generation script Signed-off-by: kramaranya <kramaranya15@gmail.com> * Update uv.lock Signed-off-by: kramaranya <kramaranya15@gmail.com> * Remove blank lines in changelog script Signed-off-by: kramaranya <kramaranya15@gmail.com> * Add make release in Makefile Signed-off-by: kramaranya <kramaranya15@gmail.com> * Refactor gen_changelog Signed-off-by: kramaranya <kramaranya15@gmail.com> * Delete CHANGELOG.md Signed-off-by: kramaranya <kramaranya15@gmail.com> * Add prepare-release CI Signed-off-by: kramaranya <kramaranya15@gmail.com> * Allow manual trigger of release CI Signed-off-by: kramaranya <kramaranya15@gmail.com> * Add release readme Signed-off-by: kramaranya <kramaranya15@gmail.com> * Update chnagelog parser in CI Signed-off-by: kramaranya <kramaranya15@gmail.com> * skip and check cherry pick for new release Signed-off-by: kramaranya <kramaranya15@gmail.com> * add script to update the version Signed-off-by: kramaranya <kramaranya15@gmail.com> * Update make release script Signed-off-by: kramaranya <kramaranya15@gmail.com> * Use make test-python in release CI Signed-off-by: kramaranya <kramaranya15@gmail.com> * Move prepare release CI to one a single workflow Signed-off-by: kramaranya <kramaranya15@gmail.com> * Remove uv.sync from RELEASE.md Signed-off-by: kramaranya <kramaranya15@gmail.com> * Upload artifacts to GitHub Release Signed-off-by: kramaranya <kramaranya15@gmail.com> * Run release CI against release-* branch Signed-off-by: kramaranya <kramaranya15@gmail.com> * Use X.Y.Z versioning in Kubeflow SDK Signed-off-by: kramaranya <kramaranya15@gmail.com> * Use single workflow for release Signed-off-by: kramaranya <kramaranya15@gmail.com> * Remove Release types from RELEASE.md Signed-off-by: kramaranya <kramaranya15@gmail.com> * Add a note about older minor series patch release Signed-off-by: kramaranya <kramaranya15@gmail.com> * Directly checkout release branch with ref input Signed-off-by: kramaranya <kramaranya15@gmail.com> * Checkout release branch with ref input Signed-off-by: kramaranya <kramaranya15@gmail.com> * Update extract changelog job Signed-off-by: kramaranya <kramaranya15@gmail.com> * Update the name of github release Signed-off-by: kramaranya <kramaranya15@gmail.com> * Remove full changelog line Signed-off-by: kramaranya <kramaranya15@gmail.com> * Fix ruff issue Signed-off-by: kramaranya <kramaranya15@gmail.com> * Don't use link for authors in changelog Signed-off-by: kramaranya <kramaranya15@gmail.com> * Remove chnagelog for RCs Signed-off-by: kramaranya <kramaranya15@gmail.com> --------- Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: kramaranya <kramaranya15@gmail.com>
Signed-off-by: Brian Gallagher <briangal@gmail.com>
Signed-off-by: Brian Gallagher <briangal@gmail.com>
…just accepting upstream
…ream chore: sync upstream changes (version, README, gen-changelog script)
sync: Merge upstream/main into main
fix: sync merge upstream/main into main
sync: Merge upstream/main into main
Pull Request Test Coverage Report for Build 19370620691Details
💛 - Coveralls |
36fbdc3 to
716dea7
Compare
716dea7 to
8a3f570
Compare
|
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Which issue(s) this PR fixes (optional, in
Fixes #<issue number>, #<issue number>, ...format, will close the issue(s) when PR gets merged):Fixes #
Checklist: