Skip to content

fix(cloud-edge-llm): fix environment setup crashes and dependency typos#394

Open
rakshaak29 wants to merge 8 commits intokubeedge:mainfrom
rakshaak29:fix/cloud-edge-env-setup
Open

fix(cloud-edge-llm): fix environment setup crashes and dependency typos#394
rakshaak29 wants to merge 8 commits intokubeedge:mainfrom
rakshaak29:fix/cloud-edge-env-setup

Conversation

@rakshaak29
Copy link
Copy Markdown

Resolves #393

Description

This PR resolves critical setup failure issues in the cloud-edge-collaborative-inference-for-llm example that blocked macOS and CPU-only users from installing dependencies and running the benchmark.

Changes Made:

  • requirements.txt:
    • Removed vllm (CUDA/Linux only) to make it an optional, conditionally loaded backend.
    • Corrected datamodel_code_generator to datamodel-code-generator.
    • Added missing runtime dependency retry.
  • test_queryrouting.yaml:
    • Commented out the vllm backend in the default configuration so the example executes successfully out-of-the-box falling back to huggingface.

Testing Done

  • Validated pip install -r requirements.txt completes successfully without CUDA.
  • Ensured CloudModel successfully runs utilizing the imported retry mechanics without ImportError.

@kubeedge-bot
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: rakshaak29
To complete the pull request process, please assign moorezheng after the PR has been reviewed.
You can assign the PR to them by writing /assign @moorezheng in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubeedge-bot kubeedge-bot requested review from Poorunga and hsj576 April 12, 2026 17:32
@kubeedge-bot kubeedge-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Apr 12, 2026
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the portability of LLM examples by replacing absolute paths with relative ones, updating the BaseModel to use a public Hugging Face model ID, and implementing dynamic device selection (CUDA/MPS/CPU). It also updates project dependencies and ignores local development artifacts. Review feedback suggests refining the MPS availability check for better compatibility, deduplicating the model identifier string, and specifying a minimum version for the torch dependency.

rakshaak29 added 5 commits April 12, 2026 23:05
…ix basemodel

- Replace all hardcoded absolute paths with relative paths in YAML configs
- Use HuggingFace Hub model ID (Qwen/Qwen2.5-0.5B-Instruct) instead of local path
- Add missing preprocess() method to BaseModel (required by sedna)
- Add MPS/CPU device detection for macOS compatibility
- Update .gitignore to exclude local dev artifacts

Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
@rakshaak29 rakshaak29 force-pushed the fix/cloud-edge-env-setup branch from fc45dbf to 67e1eab Compare April 12, 2026 17:35
rakshaak29 added 3 commits April 12, 2026 23:07
…r torch versions

Signed-off-by: rakshaak29 <rakshak29@gmail.com>
…ainability

Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Signed-off-by: rakshaak29 <rakshak29@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Cloud-Edge LLM example setup fails on macOS/CPU environments due to requirements.txt

2 participants