Skip to content

[Bug]: Cloud-Edge LLM example setup fails on macOS/CPU environments due to requirements.txt #393

@rakshaak29

Description

@rakshaak29

Describe the bug
The cloud-edge-collaborative-inference-for-llm example cannot be installed or run on macOS or CPU-only Linux machines. Running pip install -r requirements.txt fails completely.

The following issues were identified in the environment configuration:

  1. vllm is listed as a hard dependency but is strictly Linux/CUDA only.
  2. datamodel_code_generator is misspelled (PyPI uses hyphens, not underscores).
  3. api_llm.py relies on retry, but it is missing from requirements.txt.
  4. test_queryrouting.yaml default config runs vllm unconditionally instead of just huggingface.

Steps to reproduce

  1. Clone the ianvs repository on a macOS or CPU-only environment.
  2. Navigate to examples/cloud-edge-collaborative-inference-for-llm/.
  3. Run pip install -r requirements.txt.
  4. Observe installation failure for vllm and dependency errors.

Expected behavior
The requirements.txt should install cross-platform dependencies so the example runs out-of-the-box using the universally compatible huggingface backend. vllm should be treated as an optional backend parameter for Linux/CUDA users.

Metadata

Metadata

Assignees

Labels

kind/bugCategorizes issue or PR as related to a bug.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions