Error when deploying in Kubernetes: Model path does not exist #153
Description
Hello.
I am trying to run llama-gpt on Kubernetes, with manifests based on the ones provided in the repo, but I can´t get the api container to start.
It fails with error │ ValueError: Model path does not exist: /models/llama-2-7b-chat.bin
I am supposed to download the model manually?
│ │
│ Using /usr/local/lib/python3.11/site-packages │
│ Finished processing dependencies for llama-cpp-python==0.1.77 │
│ Initializing server with: │
│ Batch size: 2096 │
│ Number of CPU threads: 8 │
│ Number of GPU layers: 0 │
│ Context window: 4096 │
│ /usr/local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:126: UserWarning: Field "model_alias" has conflict with protected namespace "model_". │
│ │
│ You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ('settings_',)`. │
│ warnings.warn( │
│ Traceback (most recent call last): │
│ File "<frozen runpy>", line 198, in _run_module_as_main │
│ File "<frozen runpy>", line 88, in _run_code │
│ File "/app/llama_cpp/server/__main__.py", line 46, in <module> │
│ app = create_app(settings=settings) │
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ │
│ File "/app/llama_cpp/server/app.py", line 313, in create_app │
│ llama = llama_cpp.Llama( │
│ ^^^^^^^^^^^^^^^^ │
│ File "/app/llama_cpp/llama.py", line 308, in __init__ │
│ raise ValueError(f"Model path does not exist: {model_path}") │
│ ValueError: Model path does not exist: /models/llama-2-7b-chat.bin │
│ Exception ignored in: <function Llama.__del__ at 0x7ff10d6a6ca0> │
│ Traceback (most recent call last): │
│ File "/app/llama_cpp/llama.py", line 1507, in __del__ │
│ if self.model is not None: │
│ ^^^^^^^^^^ │
│ AttributeError: 'Llama' object has no attribute 'model'