========================================================
ToshLLM 0.82.3 — server start (x86_64)
engine : bundled (official)
model : Qwen3.6-35B-A3B-UD-Q4_K_S.gguf
GPUs detected:
[0] AMD Radeon RX 6600 XT · 7 GB
GPU select: default (macOS picks) | force-VRAM-buffers: no
settings: ngl=99 ncmoe=32 ctx=16384 fa=auto ctk=q8_0 ctv=q8_0 cacheRAM=8192 concurrencyDisable=true
env: GGML_METAL_CONCURRENCY_DISABLE=1 GGML_METAL_VRAM_RESERVE_MB=1024 TOSH_FA_AMD=1 GGML_SCHED_PREFETCH_EXPERTS=1 GGML_CPU_NO_REPACK=1
args: -m /Users/matthew/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf -ngl 99 -c 16384 -t 6 -fa 1 --host 127.0.0.1 --port 8080 --n-cpu-moe 32 --no-mmap --jinja -ctk q8_0 -ctv q8_0 --mlock --cache-ram 8192 --cache-reuse 256 --parallel 1 --path /Applications/ToshLLM.app/Contents/Resources/test-ui
========================================================
ggml_metal: probed SIMD-group width = 32 (32 = Apple/AMD RDNA, 64 = AMD GCN/Vega)
0.00.081.143 I cmn common_param: common_params_print_info: ToshLLM engine 0.82.3 (app 0.82.3)
0.00.081.147 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.082.273 I srv load_model: loading model '/Users/matthew/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf'
0.12.486.944 W srv load_model: cache_reuse is not supported by this context, it will be disabled
0.12.565.038 I srv load_model: initializing, n_slots = 1, n_ctx_slot = 16384, kv_unified = 'false'
0.12.605.969 I srv init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
0.12.606.127 I srv llama_server: model loaded
0.12.606.132 I srv llama_server: listening on http://127.0.0.1:8080
0.28.640.007 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
0.28.640.091 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
0.37.010.314 I slot print_timing: id 0 | task 0 | n_decoded = 100, tg = 18.98 t/s, tg_3s = 18.98 t/s
0.40.053.149 I slot print_timing: id 0 | task 0 | n_decoded = 158, tg = 19.01 t/s, tg_3s = 19.06 t/s
0.43.075.265 I slot print_timing: id 0 | task 0 | n_decoded = 216, tg = 19.06 t/s, tg_3s = 19.19 t/s
0.46.085.805 I slot print_timing: id 0 | task 0 | n_decoded = 273, tg = 19.03 t/s, tg_3s = 18.93 t/s
0.49.086.462 I slot print_timing: id 0 | task 0 | n_decoded = 331, tg = 19.08 t/s, tg_3s = 19.33 t/s
0.52.104.478 I slot print_timing: id 0 | task 0 | n_decoded = 389, tg = 19.10 t/s, tg_3s = 19.22 t/s
0.55.151.330 I slot print_timing: id 0 | task 0 | n_decoded = 447, tg = 19.09 t/s, tg_3s = 19.04 t/s
0.58.169.989 I slot print_timing: id 0 | task 0 | n_decoded = 504, tg = 19.07 t/s, tg_3s = 18.88 t/s
1.01.182.065 I slot print_timing: id 0 | task 0 | n_decoded = 561, tg = 19.05 t/s, tg_3s = 18.92 t/s
1.04.216.727 I slot print_timing: id 0 | task 0 | n_decoded = 619, tg = 19.06 t/s, tg_3s = 19.11 t/s
1.07.235.813 I slot print_timing: id 0 | task 0 | n_decoded = 677, tg = 19.07 t/s, tg_3s = 19.21 t/s
1.10.245.879 I slot print_timing: id 0 | task 0 | n_decoded = 734, tg = 19.06 t/s, tg_3s = 18.94 t/s
1.13.270.070 I slot print_timing: id 0 | task 0 | n_decoded = 791, tg = 19.05 t/s, tg_3s = 18.85 t/s
1.16.309.000 I slot print_timing: id 0 | task 0 | n_decoded = 849, tg = 19.05 t/s, tg_3s = 19.09 t/s
1.19.328.487 I slot print_timing: id 0 | task 0 | n_decoded = 906, tg = 19.04 t/s, tg_3s = 18.88 t/s
1.22.362.194 I slot print_timing: id 0 | task 0 | n_decoded = 964, tg = 19.04 t/s, tg_3s = 19.12 t/s
1.25.395.344 I slot print_timing: id 0 | task 0 | n_decoded = 1021, tg = 19.03 t/s, tg_3s = 18.79 t/s
1.28.409.007 I slot print_timing: id 0 | task 0 | n_decoded = 1078, tg = 19.02 t/s, tg_3s = 18.91 t/s
1.31.455.157 I slot print_timing: id 0 | task 0 | n_decoded = 1135, tg = 19.01 t/s, tg_3s = 18.71 t/s
1.34.470.864 I slot print_timing: id 0 | task 0 | n_decoded = 1192, tg = 19.00 t/s, tg_3s = 18.90 t/s
1.37.518.549 I slot print_timing: id 0 | task 0 | n_decoded = 1249, tg = 18.99 t/s, tg_3s = 18.70 t/s
1.40.535.893 I slot print_timing: id 0 | task 0 | n_decoded = 1306, tg = 18.98 t/s, tg_3s = 18.89 t/s
1.43.535.964 I slot print_timing: id 0 | task 0 | n_decoded = 1362, tg = 18.97 t/s, tg_3s = 18.67 t/s
1.46.554.780 I slot print_timing: id 0 | task 0 | n_decoded = 1419, tg = 18.97 t/s, tg_3s = 18.88 t/s
1.49.568.745 I slot print_timing: id 0 | task 0 | n_decoded = 1476, tg = 18.96 t/s, tg_3s = 18.91 t/s
1.52.601.426 I slot print_timing: id 0 | task 0 | n_decoded = 1533, tg = 18.96 t/s, tg_3s = 18.80 t/s
1.55.638.266 I slot print_timing: id 0 | task 0 | n_decoded = 1590, tg = 18.95 t/s, tg_3s = 18.77 t/s
1.58.665.940 I slot print_timing: id 0 | task 0 | n_decoded = 1646, tg = 18.94 t/s, tg_3s = 18.50 t/s
2.01.551.923 W srv stop: cancel task, id_task = 0
2.01.606.836 I slot release: id 0 | task 0 | stop processing: n_tokens = 1871, truncated = 0
3.01.059.351 I srv operator(): operator(): cleaning up before exit...
========================================================
ToshLLM 0.82.3 — server start (x86_64)
engine : bundled (official)
model : Qwen3.6-35B-A3B-UD-Q4_K_S.gguf
GPUs detected:
[0] AMD Radeon RX 6600 XT · 7 GB
GPU select: default (macOS picks) | force-VRAM-buffers: no
settings: ngl=99 ncmoe=32 ctx=16384 fa=auto ctk=q8_0 ctv=q8_0 cacheRAM=8192 concurrencyDisable=true
env: GGML_METAL_CONCURRENCY_DISABLE=1 GGML_METAL_VRAM_RESERVE_MB=1024 TOSH_FA_AMD=1 GGML_SCHED_PREFETCH_EXPERTS=1 GGML_CPU_NO_REPACK=1
args: -m /Users/matthew/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf -ngl 99 -c 16384 -t 6 -fa 1 --host 127.0.0.1 --port 8080 --n-cpu-moe 32 --no-mmap --jinja -ctk q8_0 -ctv q8_0 --mlock --cache-ram 8192 --cache-reuse 256 --parallel 1 --path /Applications/ToshLLM.app/Contents/Resources/test-ui
========================================================
ggml_metal: probed SIMD-group width = 32 (32 = Apple/AMD RDNA, 64 = AMD GCN/Vega)
0.00.090.450 I cmn common_param: common_params_print_info: ToshLLM engine 0.82.3 (app 0.82.3)
0.00.090.455 I cmn common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.091.709 I srv load_model: loading model '/Users/matthew/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf'
0.14.145.719 W srv load_model: cache_reuse is not supported by this context, it will be disabled
0.14.253.729 I srv load_model: initializing, n_slots = 1, n_ctx_slot = 16384, kv_unified = 'false'
0.14.294.659 I srv init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
0.14.294.824 I srv llama_server: model loaded
0.14.294.830 I srv llama_server: listening on http://127.0.0.1:8080
0.41.866.643 I slot get_availabl: id 0 | task -1 | selected slot by LRU, t_last = -1
0.41.866.725 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
0.52.529.508 I slot print_timing: id 0 | task 0 | n_decoded = 100, tg = 13.32 t/s, tg_3s = 13.32 t/s
0.55.543.904 I slot print_timing: id 0 | task 0 | n_decoded = 140, tg = 13.30 t/s, tg_3s = 13.27 t/s
0.58.568.251 I slot print_timing: id 0 | task 0 | n_decoded = 180, tg = 13.29 t/s, tg_3s = 13.23 t/s
1.01.642.263 I slot print_timing: id 0 | task 0 | n_decoded = 221, tg = 13.30 t/s, tg_3s = 13.34 t/s
1.04.688.684 I slot print_timing: id 0 | task 0 | n_decoded = 261, tg = 13.27 t/s, tg_3s = 13.13 t/s
1.07.734.381 I slot print_timing: id 0 | task 0 | n_decoded = 301, tg = 13.25 t/s, tg_3s = 13.13 t/s
1.10.792.687 I slot print_timing: id 0 | task 0 | n_decoded = 341, tg = 13.23 t/s, tg_3s = 13.08 t/s
1.13.867.780 I slot print_timing: id 0 | task 0 | n_decoded = 381, tg = 13.21 t/s, tg_3s = 13.01 t/s
1.16.873.042 I slot print_timing: id 0 | task 0 | n_decoded = 420, tg = 13.19 t/s, tg_3s = 12.98 t/s
1.19.889.894 I slot print_timing: id 0 | task 0 | n_decoded = 459, tg = 13.16 t/s, tg_3s = 12.93 t/s
1.22.933.701 I slot print_timing: id 0 | task 0 | n_decoded = 499, tg = 13.16 t/s, tg_3s = 13.14 t/s
1.25.955.614 I slot print_timing: id 0 | task 0 | n_decoded = 539, tg = 13.17 t/s, tg_3s = 13.24 t/s
1.28.990.558 I slot print_timing: id 0 | task 0 | n_decoded = 579, tg = 13.17 t/s, tg_3s = 13.18 t/s
1.32.039.625 I slot print_timing: id 0 | task 0 | n_decoded = 619, tg = 13.16 t/s, tg_3s = 13.12 t/s
1.35.094.129 I slot print_timing: id 0 | task 0 | n_decoded = 659, tg = 13.16 t/s, tg_3s = 13.10 t/s
1.38.097.952 I slot print_timing: id 0 | task 0 | n_decoded = 698, tg = 13.15 t/s, tg_3s = 12.98 t/s
1.41.105.008 I slot print_timing: id 0 | task 0 | n_decoded = 737, tg = 13.14 t/s, tg_3s = 12.97 t/s
1.44.121.637 I slot print_timing: id 0 | task 0 | n_decoded = 776, tg = 13.13 t/s, tg_3s = 12.93 t/s
1.47.143.062 I slot print_timing: id 0 | task 0 | n_decoded = 815, tg = 13.12 t/s, tg_3s = 12.91 t/s
1.50.175.083 I slot print_timing: id 0 | task 0 | n_decoded = 854, tg = 13.11 t/s, tg_3s = 12.86 t/s
1.53.224.982 I slot print_timing: id 0 | task 0 | n_decoded = 893, tg = 13.09 t/s, tg_3s = 12.79 t/s
1.56.293.243 I slot print_timing: id 0 | task 0 | n_decoded = 933, tg = 13.09 t/s, tg_3s = 13.04 t/s
1.59.346.146 I slot print_timing: id 0 | task 0 | n_decoded = 973, tg = 13.09 t/s, tg_3s = 13.10 t/s
2.02.400.639 I slot print_timing: id 0 | task 0 | n_decoded = 1013, tg = 13.09 t/s, tg_3s = 13.10 t/s
2.05.472.557 I slot print_timing: id 0 | task 0 | n_decoded = 1053, tg = 13.09 t/s, tg_3s = 13.02 t/s
2.08.480.231 I slot print_timing: id 0 | task 0 | n_decoded = 1092, tg = 13.08 t/s, tg_3s = 12.97 t/s
2.11.509.026 I slot print_timing: id 0 | task 0 | n_decoded = 1131, tg = 13.08 t/s, tg_3s = 12.88 t/s
2.14.534.483 I slot print_timing: id 0 | task 0 | n_decoded = 1170, tg = 13.07 t/s, tg_3s = 12.89 t/s
2.17.573.994 I slot print_timing: id 0 | task 0 | n_decoded = 1209, tg = 13.06 t/s, tg_3s = 12.83 t/s
2.20.619.499 I slot print_timing: id 0 | task 0 | n_decoded = 1248, tg = 13.05 t/s, tg_3s = 12.81 t/s
2.23.679.092 I slot print_timing: id 0 | task 0 | n_decoded = 1287, tg = 13.05 t/s, tg_3s = 12.75 t/s
2.26.750.279 I slot print_timing: id 0 | task 0 | n_decoded = 1326, tg = 13.03 t/s, tg_3s = 12.70 t/s
2.29.759.450 I slot print_timing: id 0 | task 0 | n_decoded = 1365, tg = 13.03 t/s, tg_3s = 12.96 t/s
2.32.832.748 I slot print_timing: id 0 | task 0 | n_decoded = 1405, tg = 13.03 t/s, tg_3s = 13.02 t/s
2.35.839.417 I slot print_timing: id 0 | task 0 | n_decoded = 1444, tg = 13.03 t/s, tg_3s = 12.97 t/s
2.38.844.747 I slot print_timing: id 0 | task 0 | n_decoded = 1483, tg = 13.03 t/s, tg_3s = 12.98 t/s
2.41.864.379 I slot print_timing: id 0 | task 0 | n_decoded = 1522, tg = 13.03 t/s, tg_3s = 12.92 t/s
2.44.903.338 I slot print_timing: id 0 | task 0 | n_decoded = 1561, tg = 13.02 t/s, tg_3s = 12.83 t/s
2.47.953.704 I slot print_timing: id 0 | task 0 | n_decoded = 1600, tg = 13.02 t/s, tg_3s = 12.79 t/s
2.51.026.574 I slot print_timing: id 0 | task 0 | n_decoded = 1639, tg = 13.01 t/s, tg_3s = 12.69 t/s
2.54.034.141 I slot print_timing: id 0 | task 0 | n_decoded = 1677, tg = 13.00 t/s, tg_3s = 12.63 t/s
2.57.048.610 I slot print_timing: id 0 | task 0 | n_decoded = 1715, tg = 12.99 t/s, tg_3s = 12.61 t/s
3.00.076.853 I slot print_timing: id 0 | task 0 | n_decoded = 1753, tg = 12.98 t/s, tg_3s = 12.55 t/s
3.03.121.373 I slot print_timing: id 0 | task 0 | n_decoded = 1792, tg = 12.98 t/s, tg_3s = 12.81 t/s
3.06.142.217 I slot print_timing: id 0 | task 0 | n_decoded = 1831, tg = 12.97 t/s, tg_3s = 12.91 t/s
3.09.170.335 I slot print_timing: id 0 | task 0 | n_decoded = 1870, tg = 12.97 t/s, tg_3s = 12.88 t/s
3.12.200.766 I slot print_timing: id 0 | task 0 | n_decoded = 1909, tg = 12.97 t/s, tg_3s = 12.87 t/s
3.15.262.173 I slot print_timing: id 0 | task 0 | n_decoded = 1948, tg = 12.97 t/s, tg_3s = 12.74 t/s
3.18.320.275 I slot print_timing: id 0 | task 0 | n_decoded = 1987, tg = 12.96 t/s, tg_3s = 12.75 t/s
3.21.330.674 I slot print_timing: id 0 | task 0 | n_decoded = 2025, tg = 12.96 t/s, tg_3s = 12.62 t/s
3.24.359.073 I slot print_timing: id 0 | task 0 | n_decoded = 2063, tg = 12.95 t/s, tg_3s = 12.55 t/s
3.27.390.510 I slot print_timing: id 0 | task 0 | n_decoded = 2101, tg = 12.94 t/s, tg_3s = 12.54 t/s
3.30.426.810 I slot print_timing: id 0 | task 0 | n_decoded = 2139, tg = 12.93 t/s, tg_3s = 12.52 t/s
3.33.487.284 I slot print_timing: id 0 | task 0 | n_decoded = 2177, tg = 12.92 t/s, tg_3s = 12.42 t/s
3.36.550.140 I slot print_timing: id 0 | task 0 | n_decoded = 2216, tg = 12.92 t/s, tg_3s = 12.73 t/s
3.39.592.445 I slot print_timing: id 0 | task 0 | n_decoded = 2255, tg = 12.92 t/s, tg_3s = 12.82 t/s
3.42.635.776 I slot print_timing: id 0 | task 0 | n_decoded = 2294, tg = 12.92 t/s, tg_3s = 12.81 t/s
3.45.694.140 I slot print_timing: id 0 | task 0 | n_decoded = 2333, tg = 12.91 t/s, tg_3s = 12.75 t/s
3.48.769.781 I slot print_timing: id 0 | task 0 | n_decoded = 2372, tg = 12.91 t/s, tg_3s = 12.68 t/s
3.51.794.437 I slot print_timing: id 0 | task 0 | n_decoded = 2410, tg = 12.90 t/s, tg_3s = 12.56 t/s
3.54.827.604 I slot print_timing: id 0 | task 0 | n_decoded = 2448, tg = 12.90 t/s, tg_3s = 12.53 t/s
3.57.869.075 I slot print_timing: id 0 | task 0 | n_decoded = 2486, tg = 12.89 t/s, tg_3s = 12.49 t/s
4.00.924.808 I slot print_timing: id 0 | task 0 | n_decoded = 2524, tg = 12.88 t/s, tg_3s = 12.44 t/s
4.03.992.603 I slot print_timing: id 0 | task 0 | n_decoded = 2562, tg = 12.88 t/s, tg_3s = 12.39 t/s
4.07.011.010 I slot print_timing: id 0 | task 0 | n_decoded = 2599, tg = 12.87 t/s, tg_3s = 12.26 t/s
4.10.028.398 I slot print_timing: id 0 | task 0 | n_decoded = 2637, tg = 12.86 t/s, tg_3s = 12.59 t/s
4.13.101.402 I slot print_timing: id 0 | task 0 | n_decoded = 2676, tg = 12.86 t/s, tg_3s = 12.69 t/s
4.16.171.472 I slot print_timing: id 0 | task 0 | n_decoded = 2715, tg = 12.86 t/s, tg_3s = 12.70 t/s
4.19.174.726 I slot print_timing: id 0 | task 0 | n_decoded = 2753, tg = 12.86 t/s, tg_3s = 12.65 t/s
4.22.183.425 I slot print_timing: id 0 | task 0 | n_decoded = 2791, tg = 12.85 t/s, tg_3s = 12.63 t/s
4.25.219.506 I slot print_timing: id 0 | task 0 | n_decoded = 2829, tg = 12.85 t/s, tg_3s = 12.52 t/s
4.28.262.424 I slot print_timing: id 0 | task 0 | n_decoded = 2867, tg = 12.84 t/s, tg_3s = 12.49 t/s
4.31.330.481 I slot print_timing: id 0 | task 0 | n_decoded = 2905, tg = 12.84 t/s, tg_3s = 12.39 t/s
4.34.405.243 I slot print_timing: id 0 | task 0 | n_decoded = 2943, tg = 12.83 t/s, tg_3s = 12.36 t/s
4.37.414.465 I slot print_timing: id 0 | task 0 | n_decoded = 2980, tg = 12.82 t/s, tg_3s = 12.30 t/s
4.40.451.949 I slot print_timing: id 0 | task 0 | n_decoded = 3017, tg = 12.81 t/s, tg_3s = 12.18 t/s
4.43.477.602 I slot print_timing: id 0 | task 0 | n_decoded = 3055, tg = 12.81 t/s, tg_3s = 12.56 t/s
4.46.478.697 I slot print_timing: id 0 | task 0 | n_decoded = 3093, tg = 12.81 t/s, tg_3s = 12.66 t/s
4.49.487.177 I slot print_timing: id 0 | task 0 | n_decoded = 3131, tg = 12.81 t/s, tg_3s = 12.63 t/s
4.52.524.189 I slot print_timing: id 0 | task 0 | n_decoded = 3169, tg = 12.80 t/s, tg_3s = 12.51 t/s
4.55.564.502 I slot print_timing: id 0 | task 0 | n_decoded = 3207, tg = 12.80 t/s, tg_3s = 12.50 t/s
4.58.601.532 I slot print_timing: id 0 | task 0 | n_decoded = 3245, tg = 12.80 t/s, tg_3s = 12.51 t/s
5.01.647.287 I slot print_timing: id 0 | task 0 | n_decoded = 3282, tg = 12.79 t/s, tg_3s = 12.15 t/s
5.04.725.864 I slot print_timing: id 0 | task 0 | n_decoded = 3320, tg = 12.78 t/s, tg_3s = 12.34 t/s
5.07.779.370 I slot print_timing: id 0 | task 0 | n_decoded = 3357, tg = 12.78 t/s, tg_3s = 12.12 t/s
5.10.802.778 I slot print_timing: id 0 | task 0 | n_decoded = 3395, tg = 12.77 t/s, tg_3s = 12.57 t/s
5.13.807.368 I slot print_timing: id 0 | task 0 | n_decoded = 3433, tg = 12.77 t/s, tg_3s = 12.65 t/s
5.16.842.822 I slot print_timing: id 0 | task 0 | n_decoded = 3471, tg = 12.77 t/s, tg_3s = 12.52 t/s
5.19.890.305 I slot print_timing: id 0 | task 0 | n_decoded = 3509, tg = 12.77 t/s, tg_3s = 12.47 t/s
5.22.936.270 I slot print_timing: id 0 | task 0 | n_decoded = 3547, tg = 12.76 t/s, tg_3s = 12.48 t/s
5.26.013.205 I slot print_timing: id 0 | task 0 | n_decoded = 3585, tg = 12.76 t/s, tg_3s = 12.35 t/s
5.29.040.630 I slot print_timing: id 0 | task 0 | n_decoded = 3622, tg = 12.75 t/s, tg_3s = 12.22 t/s
5.32.078.783 I slot print_timing: id 0 | task 0 | n_decoded = 3659, tg = 12.75 t/s, tg_3s = 12.18 t/s
5.35.138.736 I slot print_timing: id 0 | task 0 | n_decoded = 3696, tg = 12.74 t/s, tg_3s = 12.09 t/s
5.38.200.064 I slot print_timing: id 0 | task 0 | n_decoded = 3733, tg = 12.73 t/s, tg_3s = 12.09 t/s
5.41.230.498 I slot print_timing: id 0 | task 0 | n_decoded = 3769, tg = 12.72 t/s, tg_3s = 11.88 t/s
5.44.282.884 I slot print_timing: id 0 | task 0 | n_decoded = 3807, tg = 12.72 t/s, tg_3s = 12.45 t/s
5.47.313.232 I slot print_timing: id 0 | task 0 | n_decoded = 3845, tg = 12.72 t/s, tg_3s = 12.54 t/s
5.50.358.909 I slot print_timing: id 0 | task 0 | n_decoded = 3883, tg = 12.72 t/s, tg_3s = 12.48 t/s
5.53.412.030 I slot print_timing: id 0 | task 0 | n_decoded = 3921, tg = 12.71 t/s, tg_3s = 12.45 t/s
5.56.418.080 I slot print_timing: id 0 | task 0 | n_decoded = 3958, tg = 12.71 t/s, tg_3s = 12.31 t/s
5.59.474.326 I slot print_timing: id 0 | task 0 | n_decoded = 3996, tg = 12.71 t/s, tg_3s = 12.43 t/s
6.02.515.882 I slot print_timing: id 0 | task 0 | n_decoded = 4033, tg = 12.70 t/s, tg_3s = 12.16 t/s
6.05.563.264 I slot print_timing: id 0 | task 0 | n_decoded = 4070, tg = 12.70 t/s, tg_3s = 12.14 t/s
6.08.625.086 I slot print_timing: id 0 | task 0 | n_decoded = 4107, tg = 12.69 t/s, tg_3s = 12.08 t/s
6.11.694.978 I slot print_timing: id 0 | task 0 | n_decoded = 4144, tg = 12.69 t/s, tg_3s = 12.05 t/s
6.14.727.625 I slot print_timing: id 0 | task 0 | n_decoded = 4180, tg = 12.68 t/s, tg_3s = 11.87 t/s
6.17.798.800 I slot print_timing: id 0 | task 0 | n_decoded = 4218, tg = 12.68 t/s, tg_3s = 12.37 t/s
6.20.856.354 I slot print_timing: id 0 | task 0 | n_decoded = 4256, tg = 12.67 t/s, tg_3s = 12.43 t/s
6.23.922.881 I slot print_timing: id 0 | task 0 | n_decoded = 4294, tg = 12.67 t/s, tg_3s = 12.39 t/s
6.26.929.535 I slot print_timing: id 0 | task 0 | n_decoded = 4331, tg = 12.67 t/s, tg_3s = 12.31 t/s
6.29.946.912 I slot print_timing: id 0 | task 0 | n_decoded = 4368, tg = 12.66 t/s, tg_3s = 12.26 t/s
6.32.976.917 I slot print_timing: id 0 | task 0 | n_decoded = 4405, tg = 12.66 t/s, tg_3s = 12.21 t/s
6.36.035.841 I slot print_timing: id 0 | task 0 | n_decoded = 4442, tg = 12.65 t/s, tg_3s = 12.10 t/s
6.39.095.748 I slot print_timing: id 0 | task 0 | n_decoded = 4479, tg = 12.65 t/s, tg_3s = 12.09 t/s
6.42.172.908 I slot print_timing: id 0 | task 0 | n_decoded = 4516, tg = 12.64 t/s, tg_3s = 12.02 t/s
6.45.196.060 I slot print_timing: id 0 | task 0 | n_decoded = 4552, tg = 12.64 t/s, tg_3s = 11.91 t/s
6.48.269.590 I slot print_timing: id 0 | task 0 | n_decoded = 4588, tg = 12.63 t/s, tg_3s = 11.71 t/s
6.51.334.403 I slot print_timing: id 0 | task 0 | n_decoded = 4626, tg = 12.63 t/s, tg_3s = 12.40 t/s
6.54.410.759 I slot print_timing: id 0 | task 0 | n_decoded = 4664, tg = 12.63 t/s, tg_3s = 12.35 t/s
6.57.412.470 I slot print_timing: id 0 | task 0 | n_decoded = 4701, tg = 12.62 t/s, tg_3s = 12.33 t/s
7.00.424.732 I slot print_timing: id 0 | task 0 | n_decoded = 4738, tg = 12.62 t/s, tg_3s = 12.28 t/s
7.03.431.519 I slot print_timing: id 0 | task 0 | n_decoded = 4775, tg = 12.62 t/s, tg_3s = 12.31 t/s
7.06.457.939 I slot print_timing: id 0 | task 0 | n_decoded = 4812, tg = 12.62 t/s, tg_3s = 12.23 t/s
7.09.514.110 I slot print_timing: id 0 | task 0 | n_decoded = 4849, tg = 12.61 t/s, tg_3s = 12.11 t/s
7.12.578.332 I slot print_timing: id 0 | task 0 | n_decoded = 4886, tg = 12.61 t/s, tg_3s = 12.07 t/s
7.15.606.924 I slot print_timing: id 0 | task 0 | n_decoded = 4922, tg = 12.60 t/s, tg_3s = 11.89 t/s
7.18.661.909 I slot print_timing: id 0 | task 0 | n_decoded = 4958, tg = 12.60 t/s, tg_3s = 11.78 t/s
7.21.665.964 I slot print_timing: id 0 | task 0 | n_decoded = 4993, tg = 12.59 t/s, tg_3s = 11.65 t/s
7.24.682.783 I slot print_timing: id 0 | task 0 | n_decoded = 5030, tg = 12.59 t/s, tg_3s = 12.26 t/s
7.27.686.353 I slot print_timing: id 0 | task 0 | n_decoded = 5067, tg = 12.58 t/s, tg_3s = 12.32 t/s
7.29.643.432 I slot print_timing: id 0 | task 0 | prompt eval time = 3153.97 ms / 171 tokens ( 18.44 ms per token, 54.22 tokens per second)
7.29.643.437 I slot print_timing: id 0 | task 0 | eval time = 404621.74 ms / 5091 tokens ( 79.48 ms per token, 12.58 tokens per second)
7.29.643.437 I slot print_timing: id 0 | task 0 | total time = 407775.71 ms / 5262 tokens
7.29.643.438 I slot print_timing: id 0 | task 0 | graphs reused = 5070
7.29.643.620 I slot release: id 0 | task 0 | stop processing: n_tokens = 5261, truncated = 0
macOS version
15.2
GPU and VRAM
AMD RX 6600 XT
Model and quantization
Qwen3.6-35B-A3B UD-Q4_K_S
What happened?
14t/s --> 19t/s Just by removing all 0-3
ATY,Henbury.Detailed Information:
toshllm.com benchmarks d062e6809933 14t/s
toshllm.com benchmarks d062e6809933 19t/s
The original file:
Server log