fix: handle llama_cpp/llguidance version incompatibility gracefully#1452
Open
octo-patch wants to merge 1 commit into
Open
fix: handle llama_cpp/llguidance version incompatibility gracefully#1452octo-patch wants to merge 1 commit into
octo-patch wants to merge 1 commit into
Conversation
When llama_cpp is installed but at a version incompatible with llguidance.llamacpp (e.g. llama-cpp-python 0.2.x with llguidance 1.5+), the module-level import llguidance.llamacpp raised an AttributeError that crashed the entire guidance import, making the library unusable even for users not relying on LlamaCpp. This change consolidates both imports into a single try/except block that catches ModuleNotFoundError, ImportError, and AttributeError, setting is_llama_cpp=False when either package is unavailable or incompatible. Users who attempt to use LlamaCpp will still receive a clear error message at instantiation time.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1384
Problem
When
llama_cppis installed but at a version incompatible withllguidance.llamacpp(e.g.
llama-cpp-python==0.2.xwithllguidance>=1.5.0), the module-levelimport llguidance.llamacppraises anAttributeError(e.g.module 'llama_cpp' has no attribute 'llama_vocab_p') that crashes the entireguidanceimport.This makes the library completely unusable even for users who are not using LlamaCpp
at all, since
guidance/models/__init__.pyimports from_llama_cpp.pyat module level.Verified on:
llama-cpp-python==0.2.7+llguidance==1.5.0:Solution
Consolidate both
import llama_cppandimport llguidance.llamacppinto a singletry/exceptblock that catchesModuleNotFoundError,ImportError, andAttributeError. When either package is missing or incompatible,is_llama_cppis set to
Falseand the rest of guidance loads normally.Users who attempt to instantiate
LlamaCppwill still receive a clear error messageat instantiation time (the existing check at the top of
LlamaCppEngine.__init__).Testing
import guidancesucceeds withllama-cpp-python==0.2.7installedpytest tests/unit/library/test_gen.py tests/unit/test_decorator.py(44 passed, 1 xfailed)