I struggled for a while to see why my the model was not loaded with the KV cache quantizaiton defined in the yaml file.
Turned out the model was actually representing the LM Studio UI and I had to define and set the "checked" to true.
|
result.llamaKCacheQuantizationType = llamaKCacheQuantizationType.checked |
I find that this breaks the definition of an open standard as defined in https://github.com/modelyaml/modelyaml if it's just another UI state leak.
I struggled for a while to see why my the model was not loaded with the KV cache quantizaiton defined in the yaml file.
Turned out the model was actually representing the LM Studio UI and I had to define and set the "checked" to true.
lmstudio-js/packages/lms-kv-config/src/conversion/llmLoadModelConfig.ts
Line 144 in 85cac63
I find that this breaks the definition of an open standard as defined in https://github.com/modelyaml/modelyaml if it's just another UI state leak.