fix(cloud-edge-llm): fix environment setup crashes and dependency typos#394
fix(cloud-edge-llm): fix environment setup crashes and dependency typos#394rakshaak29 wants to merge 8 commits intokubeedge:mainfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: rakshaak29 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
There was a problem hiding this comment.
Code Review
This pull request enhances the portability of LLM examples by replacing absolute paths with relative ones, updating the BaseModel to use a public Hugging Face model ID, and implementing dynamic device selection (CUDA/MPS/CPU). It also updates project dependencies and ignores local development artifacts. Review feedback suggests refining the MPS availability check for better compatibility, deduplicating the model identifier string, and specifying a minimum version for the torch dependency.
examples/cloud-edge-collaborative-inference-for-llm/requirements.txt
Outdated
Show resolved
Hide resolved
…ix basemodel - Replace all hardcoded absolute paths with relative paths in YAML configs - Use HuggingFace Hub model ID (Qwen/Qwen2.5-0.5B-Instruct) instead of local path - Add missing preprocess() method to BaseModel (required by sedna) - Add MPS/CPU device detection for macOS compatibility - Update .gitignore to exclude local dev artifacts Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
fc45dbf to
67e1eab
Compare
…r torch versions Signed-off-by: rakshaak29 <rakshak29@gmail.com>
…ainability Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Resolves #393
Description
This PR resolves critical setup failure issues in the
cloud-edge-collaborative-inference-for-llmexample that blocked macOS and CPU-only users from installing dependencies and running the benchmark.Changes Made:
requirements.txt:vllm(CUDA/Linux only) to make it an optional, conditionally loaded backend.datamodel_code_generatortodatamodel-code-generator.retry.test_queryrouting.yaml:vllmbackend in the default configuration so the example executes successfully out-of-the-box falling back tohuggingface.Testing Done
pip install -r requirements.txtcompletes successfully without CUDA.CloudModelsuccessfully runs utilizing the importedretrymechanics withoutImportError.