Skip to content

Commit 13fe37d

Browse files
LocalAI [bot]mudler
andauthored
chore(model gallery): 🤖 add 1 new models via gallery agent (mudler#9611)
chore(model gallery): 🤖 add new models via gallery agent Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: mudler <2420543+mudler@users.noreply.github.com>
1 parent 4916f8c commit 13fe37d

1 file changed

Lines changed: 43 additions & 0 deletions

File tree

gallery/index.yaml

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,47 @@
11
---
2+
- name: "carnice-v2-27b"
3+
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
4+
urls:
5+
- https://huggingface.co/kai-os/Carnice-V2-27b-GGUF
6+
description: |
7+
# Carnice-V2-27B for Hermes Agent
8+
9+
Carnice-V2-27B is a full merged BF16 SFT of `Qwen/Qwen3.6-27B` for Hermes-style agent traces. This repository contains the standalone merged model weights, not only a LoRA adapter.
10+
11+
## BF16 Transformers Loading Fix
12+
13+
The BF16 safetensors were republished with corrected `Qwen3_5ForConditionalGeneration` tensor prefixes. The original merge artifact accidentally serialized an extra Unsloth wrapper prefix, which caused direct HF Transformers loads to report the real weights as unexpected keys and initialize expected layers randomly. GGUF files were not affected because the GGUF conversion path normalized those prefixes.
14+
15+
## Benchmarks
16+
17+
The benchmark artifact bundle is included under `benchmarks/`. It contains the rendered graph, extracted `metrics.json`, benchmark scripts, and raw result files used to make the chart.
18+
19+
Scope note: the IFEval run is a short `limit=20` A/B smoke benchmark, not an official full leaderboard score. Held-out loss/perplexity is the exact assistant-only training-format validation metric from the SFT script. The raw BFCL two-case smoke files are included for auditability, but they are too small to use as a model-quality claim.
20+
21+
...
22+
license: "apache-2.0"
23+
tags:
24+
- llm
25+
- gguf
26+
- qwen
27+
overrides:
28+
backend: llama-cpp
29+
function:
30+
automatic_tool_parsing_fallback: true
31+
grammar:
32+
disable: true
33+
known_usecases:
34+
- chat
35+
options:
36+
- use_jinja:true
37+
parameters:
38+
model: llama-cpp/models/Carnice-V2-27b-GGUF/carnice-v2-27b-Q4_K_M.gguf
39+
template:
40+
use_tokenizer_template: true
41+
files:
42+
- filename: llama-cpp/models/Carnice-V2-27b-GGUF/carnice-v2-27b-Q4_K_M.gguf
43+
sha256: 85b7f41f22b80fce910286c2457022a067d45b91a2629046adcec0b6942ea359
44+
uri: https://huggingface.co/kai-os/Carnice-V2-27b-GGUF/resolve/main/carnice-v2-27b-Q4_K_M.gguf
245
- name: "kimi-k2.6"
346
url: "github:mudler/LocalAI/gallery/virtual.yaml@master"
447
urls:

0 commit comments

Comments
 (0)