Open
Description
Hi! Thanks for making this library!
I tried two models with it, phind-codellama-34b-v2.Q4_K_M.gguf
worked fine, but I got an error with another one:
error: ZodError: [
{
"code": "invalid_type",
"expected": "number",
"received": "undefined",
"path": [
"general",
"quantization_version"
],
"message": "Required"
}
]
model: orca-mini-v2_7b.ggmlv3.q4_0.bin
converted to GGUF with https://github.com/ggerganov/llama.cpp/blob/master/convert-llama-ggml-to-gguf.py
it was converted before this commit ggerganov/llama.cpp@ea2c85d
P.S. Do you have any plans on providing direct layer/weight data access (R/W?) Or is this library only for metadata?
Metadata
Assignees
Labels
No labels