Skip to content

Removing Henbury FB from DeviceProperties resulted in a 35% increase in token generation #44

Description

@Ardentwheel

macOS version

15.2

GPU and VRAM

AMD RX 6600 XT

Model and quantization

Qwen3.6-35B-A3B UD-Q4_K_S

What happened?

14t/s --> 19t/s Just by removing all 0-3 ATY,Henbury.

Detailed Information:
toshllm.com benchmarks d062e6809933 14t/s
toshllm.com benchmarks d062e6809933 19t/s

The original file:

<plist version="1.0">
<dict>
	<key>DeviceProperties</key>
	<dict>
		<key>Add</key>
		<dict>
			<key>PciRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x0)/Pci(0x0,0x0)/Pci(0x0,0x0)</key>
			<dict>
				<key>@0,name</key>
				<string>ATY,Henbury</string>
				<key>@1,name</key>
				<string>ATY,Henbury</string>
				<key>@2,name</key>
				<string>ATY,Henbury</string>
				<key>@3,name</key>
				<string>ATY,Henbury</string>
				<key>AAPL,slot-name</key>
				<string>Slot-1</string>
			</dict>
		</dict>
		<key>Delete</key>
		<dict/>
	</dict>
</dict>
</plist>

Server log

========================================================
 ToshLLM 0.82.3 — server start (x86_64)
 engine : bundled (official)
 model  : Qwen3.6-35B-A3B-UD-Q4_K_S.gguf
 GPUs detected:
    [0] AMD Radeon RX 6600 XT · 7 GB
 GPU select: default (macOS picks) | force-VRAM-buffers: no
 settings: ngl=99 ncmoe=32 ctx=16384 fa=auto ctk=q8_0 ctv=q8_0 cacheRAM=8192 concurrencyDisable=true
 env: GGML_METAL_CONCURRENCY_DISABLE=1 GGML_METAL_VRAM_RESERVE_MB=1024 TOSH_FA_AMD=1 GGML_SCHED_PREFETCH_EXPERTS=1 GGML_CPU_NO_REPACK=1
 args: -m /Users/matthew/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf -ngl 99 -c 16384 -t 6 -fa 1 --host 127.0.0.1 --port 8080 --n-cpu-moe 32 --no-mmap --jinja -ctk q8_0 -ctv q8_0 --mlock --cache-ram 8192 --cache-reuse 256 --parallel 1 --path /Applications/ToshLLM.app/Contents/Resources/test-ui
========================================================
ggml_metal: probed SIMD-group width = 32 (32 = Apple/AMD RDNA, 64 = AMD GCN/Vega)
0.00.081.143 I cmn  common_param: common_params_print_info: ToshLLM engine 0.82.3 (app 0.82.3)
0.00.081.147 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.082.273 I srv    load_model: loading model '/Users/matthew/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf'
0.12.486.944 W srv    load_model: cache_reuse is not supported by this context, it will be disabled
0.12.565.038 I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 16384, kv_unified = 'false'
0.12.605.969 I srv          init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
0.12.606.127 I srv  llama_server: model loaded
0.12.606.132 I srv  llama_server: listening on http://127.0.0.1:8080
0.28.640.007 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
0.28.640.091 I slot launch_slot_: id  0 | task 0 | processing task, is_child = 0
0.37.010.314 I slot print_timing: id  0 | task 0 | n_decoded =    100, tg =  18.98 t/s, tg_3s =  18.98 t/s
0.40.053.149 I slot print_timing: id  0 | task 0 | n_decoded =    158, tg =  19.01 t/s, tg_3s =  19.06 t/s
0.43.075.265 I slot print_timing: id  0 | task 0 | n_decoded =    216, tg =  19.06 t/s, tg_3s =  19.19 t/s
0.46.085.805 I slot print_timing: id  0 | task 0 | n_decoded =    273, tg =  19.03 t/s, tg_3s =  18.93 t/s
0.49.086.462 I slot print_timing: id  0 | task 0 | n_decoded =    331, tg =  19.08 t/s, tg_3s =  19.33 t/s
0.52.104.478 I slot print_timing: id  0 | task 0 | n_decoded =    389, tg =  19.10 t/s, tg_3s =  19.22 t/s
0.55.151.330 I slot print_timing: id  0 | task 0 | n_decoded =    447, tg =  19.09 t/s, tg_3s =  19.04 t/s
0.58.169.989 I slot print_timing: id  0 | task 0 | n_decoded =    504, tg =  19.07 t/s, tg_3s =  18.88 t/s
1.01.182.065 I slot print_timing: id  0 | task 0 | n_decoded =    561, tg =  19.05 t/s, tg_3s =  18.92 t/s
1.04.216.727 I slot print_timing: id  0 | task 0 | n_decoded =    619, tg =  19.06 t/s, tg_3s =  19.11 t/s
1.07.235.813 I slot print_timing: id  0 | task 0 | n_decoded =    677, tg =  19.07 t/s, tg_3s =  19.21 t/s
1.10.245.879 I slot print_timing: id  0 | task 0 | n_decoded =    734, tg =  19.06 t/s, tg_3s =  18.94 t/s
1.13.270.070 I slot print_timing: id  0 | task 0 | n_decoded =    791, tg =  19.05 t/s, tg_3s =  18.85 t/s
1.16.309.000 I slot print_timing: id  0 | task 0 | n_decoded =    849, tg =  19.05 t/s, tg_3s =  19.09 t/s
1.19.328.487 I slot print_timing: id  0 | task 0 | n_decoded =    906, tg =  19.04 t/s, tg_3s =  18.88 t/s
1.22.362.194 I slot print_timing: id  0 | task 0 | n_decoded =    964, tg =  19.04 t/s, tg_3s =  19.12 t/s
1.25.395.344 I slot print_timing: id  0 | task 0 | n_decoded =   1021, tg =  19.03 t/s, tg_3s =  18.79 t/s
1.28.409.007 I slot print_timing: id  0 | task 0 | n_decoded =   1078, tg =  19.02 t/s, tg_3s =  18.91 t/s
1.31.455.157 I slot print_timing: id  0 | task 0 | n_decoded =   1135, tg =  19.01 t/s, tg_3s =  18.71 t/s
1.34.470.864 I slot print_timing: id  0 | task 0 | n_decoded =   1192, tg =  19.00 t/s, tg_3s =  18.90 t/s
1.37.518.549 I slot print_timing: id  0 | task 0 | n_decoded =   1249, tg =  18.99 t/s, tg_3s =  18.70 t/s
1.40.535.893 I slot print_timing: id  0 | task 0 | n_decoded =   1306, tg =  18.98 t/s, tg_3s =  18.89 t/s
1.43.535.964 I slot print_timing: id  0 | task 0 | n_decoded =   1362, tg =  18.97 t/s, tg_3s =  18.67 t/s
1.46.554.780 I slot print_timing: id  0 | task 0 | n_decoded =   1419, tg =  18.97 t/s, tg_3s =  18.88 t/s
1.49.568.745 I slot print_timing: id  0 | task 0 | n_decoded =   1476, tg =  18.96 t/s, tg_3s =  18.91 t/s
1.52.601.426 I slot print_timing: id  0 | task 0 | n_decoded =   1533, tg =  18.96 t/s, tg_3s =  18.80 t/s
1.55.638.266 I slot print_timing: id  0 | task 0 | n_decoded =   1590, tg =  18.95 t/s, tg_3s =  18.77 t/s
1.58.665.940 I slot print_timing: id  0 | task 0 | n_decoded =   1646, tg =  18.94 t/s, tg_3s =  18.50 t/s
2.01.551.923 W srv          stop: cancel task, id_task = 0
2.01.606.836 I slot      release: id  0 | task 0 | stop processing: n_tokens = 1871, truncated = 0
3.01.059.351 I srv    operator(): operator(): cleaning up before exit...
========================================================
 ToshLLM 0.82.3 — server start (x86_64)
 engine : bundled (official)
 model  : Qwen3.6-35B-A3B-UD-Q4_K_S.gguf
 GPUs detected:
    [0] AMD Radeon RX 6600 XT · 7 GB
 GPU select: default (macOS picks) | force-VRAM-buffers: no
 settings: ngl=99 ncmoe=32 ctx=16384 fa=auto ctk=q8_0 ctv=q8_0 cacheRAM=8192 concurrencyDisable=true
 env: GGML_METAL_CONCURRENCY_DISABLE=1 GGML_METAL_VRAM_RESERVE_MB=1024 TOSH_FA_AMD=1 GGML_SCHED_PREFETCH_EXPERTS=1 GGML_CPU_NO_REPACK=1
 args: -m /Users/matthew/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf -ngl 99 -c 16384 -t 6 -fa 1 --host 127.0.0.1 --port 8080 --n-cpu-moe 32 --no-mmap --jinja -ctk q8_0 -ctv q8_0 --mlock --cache-ram 8192 --cache-reuse 256 --parallel 1 --path /Applications/ToshLLM.app/Contents/Resources/test-ui
========================================================
ggml_metal: probed SIMD-group width = 32 (32 = Apple/AMD RDNA, 64 = AMD GCN/Vega)
0.00.090.450 I cmn  common_param: common_params_print_info: ToshLLM engine 0.82.3 (app 0.82.3)
0.00.090.455 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.091.709 I srv    load_model: loading model '/Users/matthew/models/Qwen3.6-35B-A3B-UD-Q4_K_S.gguf'
0.14.145.719 W srv    load_model: cache_reuse is not supported by this context, it will be disabled
0.14.253.729 I srv    load_model: initializing, n_slots = 1, n_ctx_slot = 16384, kv_unified = 'false'
0.14.294.659 I srv          init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
0.14.294.824 I srv  llama_server: model loaded
0.14.294.830 I srv  llama_server: listening on http://127.0.0.1:8080
0.41.866.643 I slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
0.41.866.725 I slot launch_slot_: id  0 | task 0 | processing task, is_child = 0
0.52.529.508 I slot print_timing: id  0 | task 0 | n_decoded =    100, tg =  13.32 t/s, tg_3s =  13.32 t/s
0.55.543.904 I slot print_timing: id  0 | task 0 | n_decoded =    140, tg =  13.30 t/s, tg_3s =  13.27 t/s
0.58.568.251 I slot print_timing: id  0 | task 0 | n_decoded =    180, tg =  13.29 t/s, tg_3s =  13.23 t/s
1.01.642.263 I slot print_timing: id  0 | task 0 | n_decoded =    221, tg =  13.30 t/s, tg_3s =  13.34 t/s
1.04.688.684 I slot print_timing: id  0 | task 0 | n_decoded =    261, tg =  13.27 t/s, tg_3s =  13.13 t/s
1.07.734.381 I slot print_timing: id  0 | task 0 | n_decoded =    301, tg =  13.25 t/s, tg_3s =  13.13 t/s
1.10.792.687 I slot print_timing: id  0 | task 0 | n_decoded =    341, tg =  13.23 t/s, tg_3s =  13.08 t/s
1.13.867.780 I slot print_timing: id  0 | task 0 | n_decoded =    381, tg =  13.21 t/s, tg_3s =  13.01 t/s
1.16.873.042 I slot print_timing: id  0 | task 0 | n_decoded =    420, tg =  13.19 t/s, tg_3s =  12.98 t/s
1.19.889.894 I slot print_timing: id  0 | task 0 | n_decoded =    459, tg =  13.16 t/s, tg_3s =  12.93 t/s
1.22.933.701 I slot print_timing: id  0 | task 0 | n_decoded =    499, tg =  13.16 t/s, tg_3s =  13.14 t/s
1.25.955.614 I slot print_timing: id  0 | task 0 | n_decoded =    539, tg =  13.17 t/s, tg_3s =  13.24 t/s
1.28.990.558 I slot print_timing: id  0 | task 0 | n_decoded =    579, tg =  13.17 t/s, tg_3s =  13.18 t/s
1.32.039.625 I slot print_timing: id  0 | task 0 | n_decoded =    619, tg =  13.16 t/s, tg_3s =  13.12 t/s
1.35.094.129 I slot print_timing: id  0 | task 0 | n_decoded =    659, tg =  13.16 t/s, tg_3s =  13.10 t/s
1.38.097.952 I slot print_timing: id  0 | task 0 | n_decoded =    698, tg =  13.15 t/s, tg_3s =  12.98 t/s
1.41.105.008 I slot print_timing: id  0 | task 0 | n_decoded =    737, tg =  13.14 t/s, tg_3s =  12.97 t/s
1.44.121.637 I slot print_timing: id  0 | task 0 | n_decoded =    776, tg =  13.13 t/s, tg_3s =  12.93 t/s
1.47.143.062 I slot print_timing: id  0 | task 0 | n_decoded =    815, tg =  13.12 t/s, tg_3s =  12.91 t/s
1.50.175.083 I slot print_timing: id  0 | task 0 | n_decoded =    854, tg =  13.11 t/s, tg_3s =  12.86 t/s
1.53.224.982 I slot print_timing: id  0 | task 0 | n_decoded =    893, tg =  13.09 t/s, tg_3s =  12.79 t/s
1.56.293.243 I slot print_timing: id  0 | task 0 | n_decoded =    933, tg =  13.09 t/s, tg_3s =  13.04 t/s
1.59.346.146 I slot print_timing: id  0 | task 0 | n_decoded =    973, tg =  13.09 t/s, tg_3s =  13.10 t/s
2.02.400.639 I slot print_timing: id  0 | task 0 | n_decoded =   1013, tg =  13.09 t/s, tg_3s =  13.10 t/s
2.05.472.557 I slot print_timing: id  0 | task 0 | n_decoded =   1053, tg =  13.09 t/s, tg_3s =  13.02 t/s
2.08.480.231 I slot print_timing: id  0 | task 0 | n_decoded =   1092, tg =  13.08 t/s, tg_3s =  12.97 t/s
2.11.509.026 I slot print_timing: id  0 | task 0 | n_decoded =   1131, tg =  13.08 t/s, tg_3s =  12.88 t/s
2.14.534.483 I slot print_timing: id  0 | task 0 | n_decoded =   1170, tg =  13.07 t/s, tg_3s =  12.89 t/s
2.17.573.994 I slot print_timing: id  0 | task 0 | n_decoded =   1209, tg =  13.06 t/s, tg_3s =  12.83 t/s
2.20.619.499 I slot print_timing: id  0 | task 0 | n_decoded =   1248, tg =  13.05 t/s, tg_3s =  12.81 t/s
2.23.679.092 I slot print_timing: id  0 | task 0 | n_decoded =   1287, tg =  13.05 t/s, tg_3s =  12.75 t/s
2.26.750.279 I slot print_timing: id  0 | task 0 | n_decoded =   1326, tg =  13.03 t/s, tg_3s =  12.70 t/s
2.29.759.450 I slot print_timing: id  0 | task 0 | n_decoded =   1365, tg =  13.03 t/s, tg_3s =  12.96 t/s
2.32.832.748 I slot print_timing: id  0 | task 0 | n_decoded =   1405, tg =  13.03 t/s, tg_3s =  13.02 t/s
2.35.839.417 I slot print_timing: id  0 | task 0 | n_decoded =   1444, tg =  13.03 t/s, tg_3s =  12.97 t/s
2.38.844.747 I slot print_timing: id  0 | task 0 | n_decoded =   1483, tg =  13.03 t/s, tg_3s =  12.98 t/s
2.41.864.379 I slot print_timing: id  0 | task 0 | n_decoded =   1522, tg =  13.03 t/s, tg_3s =  12.92 t/s
2.44.903.338 I slot print_timing: id  0 | task 0 | n_decoded =   1561, tg =  13.02 t/s, tg_3s =  12.83 t/s
2.47.953.704 I slot print_timing: id  0 | task 0 | n_decoded =   1600, tg =  13.02 t/s, tg_3s =  12.79 t/s
2.51.026.574 I slot print_timing: id  0 | task 0 | n_decoded =   1639, tg =  13.01 t/s, tg_3s =  12.69 t/s
2.54.034.141 I slot print_timing: id  0 | task 0 | n_decoded =   1677, tg =  13.00 t/s, tg_3s =  12.63 t/s
2.57.048.610 I slot print_timing: id  0 | task 0 | n_decoded =   1715, tg =  12.99 t/s, tg_3s =  12.61 t/s
3.00.076.853 I slot print_timing: id  0 | task 0 | n_decoded =   1753, tg =  12.98 t/s, tg_3s =  12.55 t/s
3.03.121.373 I slot print_timing: id  0 | task 0 | n_decoded =   1792, tg =  12.98 t/s, tg_3s =  12.81 t/s
3.06.142.217 I slot print_timing: id  0 | task 0 | n_decoded =   1831, tg =  12.97 t/s, tg_3s =  12.91 t/s
3.09.170.335 I slot print_timing: id  0 | task 0 | n_decoded =   1870, tg =  12.97 t/s, tg_3s =  12.88 t/s
3.12.200.766 I slot print_timing: id  0 | task 0 | n_decoded =   1909, tg =  12.97 t/s, tg_3s =  12.87 t/s
3.15.262.173 I slot print_timing: id  0 | task 0 | n_decoded =   1948, tg =  12.97 t/s, tg_3s =  12.74 t/s
3.18.320.275 I slot print_timing: id  0 | task 0 | n_decoded =   1987, tg =  12.96 t/s, tg_3s =  12.75 t/s
3.21.330.674 I slot print_timing: id  0 | task 0 | n_decoded =   2025, tg =  12.96 t/s, tg_3s =  12.62 t/s
3.24.359.073 I slot print_timing: id  0 | task 0 | n_decoded =   2063, tg =  12.95 t/s, tg_3s =  12.55 t/s
3.27.390.510 I slot print_timing: id  0 | task 0 | n_decoded =   2101, tg =  12.94 t/s, tg_3s =  12.54 t/s
3.30.426.810 I slot print_timing: id  0 | task 0 | n_decoded =   2139, tg =  12.93 t/s, tg_3s =  12.52 t/s
3.33.487.284 I slot print_timing: id  0 | task 0 | n_decoded =   2177, tg =  12.92 t/s, tg_3s =  12.42 t/s
3.36.550.140 I slot print_timing: id  0 | task 0 | n_decoded =   2216, tg =  12.92 t/s, tg_3s =  12.73 t/s
3.39.592.445 I slot print_timing: id  0 | task 0 | n_decoded =   2255, tg =  12.92 t/s, tg_3s =  12.82 t/s
3.42.635.776 I slot print_timing: id  0 | task 0 | n_decoded =   2294, tg =  12.92 t/s, tg_3s =  12.81 t/s
3.45.694.140 I slot print_timing: id  0 | task 0 | n_decoded =   2333, tg =  12.91 t/s, tg_3s =  12.75 t/s
3.48.769.781 I slot print_timing: id  0 | task 0 | n_decoded =   2372, tg =  12.91 t/s, tg_3s =  12.68 t/s
3.51.794.437 I slot print_timing: id  0 | task 0 | n_decoded =   2410, tg =  12.90 t/s, tg_3s =  12.56 t/s
3.54.827.604 I slot print_timing: id  0 | task 0 | n_decoded =   2448, tg =  12.90 t/s, tg_3s =  12.53 t/s
3.57.869.075 I slot print_timing: id  0 | task 0 | n_decoded =   2486, tg =  12.89 t/s, tg_3s =  12.49 t/s
4.00.924.808 I slot print_timing: id  0 | task 0 | n_decoded =   2524, tg =  12.88 t/s, tg_3s =  12.44 t/s
4.03.992.603 I slot print_timing: id  0 | task 0 | n_decoded =   2562, tg =  12.88 t/s, tg_3s =  12.39 t/s
4.07.011.010 I slot print_timing: id  0 | task 0 | n_decoded =   2599, tg =  12.87 t/s, tg_3s =  12.26 t/s
4.10.028.398 I slot print_timing: id  0 | task 0 | n_decoded =   2637, tg =  12.86 t/s, tg_3s =  12.59 t/s
4.13.101.402 I slot print_timing: id  0 | task 0 | n_decoded =   2676, tg =  12.86 t/s, tg_3s =  12.69 t/s
4.16.171.472 I slot print_timing: id  0 | task 0 | n_decoded =   2715, tg =  12.86 t/s, tg_3s =  12.70 t/s
4.19.174.726 I slot print_timing: id  0 | task 0 | n_decoded =   2753, tg =  12.86 t/s, tg_3s =  12.65 t/s
4.22.183.425 I slot print_timing: id  0 | task 0 | n_decoded =   2791, tg =  12.85 t/s, tg_3s =  12.63 t/s
4.25.219.506 I slot print_timing: id  0 | task 0 | n_decoded =   2829, tg =  12.85 t/s, tg_3s =  12.52 t/s
4.28.262.424 I slot print_timing: id  0 | task 0 | n_decoded =   2867, tg =  12.84 t/s, tg_3s =  12.49 t/s
4.31.330.481 I slot print_timing: id  0 | task 0 | n_decoded =   2905, tg =  12.84 t/s, tg_3s =  12.39 t/s
4.34.405.243 I slot print_timing: id  0 | task 0 | n_decoded =   2943, tg =  12.83 t/s, tg_3s =  12.36 t/s
4.37.414.465 I slot print_timing: id  0 | task 0 | n_decoded =   2980, tg =  12.82 t/s, tg_3s =  12.30 t/s
4.40.451.949 I slot print_timing: id  0 | task 0 | n_decoded =   3017, tg =  12.81 t/s, tg_3s =  12.18 t/s
4.43.477.602 I slot print_timing: id  0 | task 0 | n_decoded =   3055, tg =  12.81 t/s, tg_3s =  12.56 t/s
4.46.478.697 I slot print_timing: id  0 | task 0 | n_decoded =   3093, tg =  12.81 t/s, tg_3s =  12.66 t/s
4.49.487.177 I slot print_timing: id  0 | task 0 | n_decoded =   3131, tg =  12.81 t/s, tg_3s =  12.63 t/s
4.52.524.189 I slot print_timing: id  0 | task 0 | n_decoded =   3169, tg =  12.80 t/s, tg_3s =  12.51 t/s
4.55.564.502 I slot print_timing: id  0 | task 0 | n_decoded =   3207, tg =  12.80 t/s, tg_3s =  12.50 t/s
4.58.601.532 I slot print_timing: id  0 | task 0 | n_decoded =   3245, tg =  12.80 t/s, tg_3s =  12.51 t/s
5.01.647.287 I slot print_timing: id  0 | task 0 | n_decoded =   3282, tg =  12.79 t/s, tg_3s =  12.15 t/s
5.04.725.864 I slot print_timing: id  0 | task 0 | n_decoded =   3320, tg =  12.78 t/s, tg_3s =  12.34 t/s
5.07.779.370 I slot print_timing: id  0 | task 0 | n_decoded =   3357, tg =  12.78 t/s, tg_3s =  12.12 t/s
5.10.802.778 I slot print_timing: id  0 | task 0 | n_decoded =   3395, tg =  12.77 t/s, tg_3s =  12.57 t/s
5.13.807.368 I slot print_timing: id  0 | task 0 | n_decoded =   3433, tg =  12.77 t/s, tg_3s =  12.65 t/s
5.16.842.822 I slot print_timing: id  0 | task 0 | n_decoded =   3471, tg =  12.77 t/s, tg_3s =  12.52 t/s
5.19.890.305 I slot print_timing: id  0 | task 0 | n_decoded =   3509, tg =  12.77 t/s, tg_3s =  12.47 t/s
5.22.936.270 I slot print_timing: id  0 | task 0 | n_decoded =   3547, tg =  12.76 t/s, tg_3s =  12.48 t/s
5.26.013.205 I slot print_timing: id  0 | task 0 | n_decoded =   3585, tg =  12.76 t/s, tg_3s =  12.35 t/s
5.29.040.630 I slot print_timing: id  0 | task 0 | n_decoded =   3622, tg =  12.75 t/s, tg_3s =  12.22 t/s
5.32.078.783 I slot print_timing: id  0 | task 0 | n_decoded =   3659, tg =  12.75 t/s, tg_3s =  12.18 t/s
5.35.138.736 I slot print_timing: id  0 | task 0 | n_decoded =   3696, tg =  12.74 t/s, tg_3s =  12.09 t/s
5.38.200.064 I slot print_timing: id  0 | task 0 | n_decoded =   3733, tg =  12.73 t/s, tg_3s =  12.09 t/s
5.41.230.498 I slot print_timing: id  0 | task 0 | n_decoded =   3769, tg =  12.72 t/s, tg_3s =  11.88 t/s
5.44.282.884 I slot print_timing: id  0 | task 0 | n_decoded =   3807, tg =  12.72 t/s, tg_3s =  12.45 t/s
5.47.313.232 I slot print_timing: id  0 | task 0 | n_decoded =   3845, tg =  12.72 t/s, tg_3s =  12.54 t/s
5.50.358.909 I slot print_timing: id  0 | task 0 | n_decoded =   3883, tg =  12.72 t/s, tg_3s =  12.48 t/s
5.53.412.030 I slot print_timing: id  0 | task 0 | n_decoded =   3921, tg =  12.71 t/s, tg_3s =  12.45 t/s
5.56.418.080 I slot print_timing: id  0 | task 0 | n_decoded =   3958, tg =  12.71 t/s, tg_3s =  12.31 t/s
5.59.474.326 I slot print_timing: id  0 | task 0 | n_decoded =   3996, tg =  12.71 t/s, tg_3s =  12.43 t/s
6.02.515.882 I slot print_timing: id  0 | task 0 | n_decoded =   4033, tg =  12.70 t/s, tg_3s =  12.16 t/s
6.05.563.264 I slot print_timing: id  0 | task 0 | n_decoded =   4070, tg =  12.70 t/s, tg_3s =  12.14 t/s
6.08.625.086 I slot print_timing: id  0 | task 0 | n_decoded =   4107, tg =  12.69 t/s, tg_3s =  12.08 t/s
6.11.694.978 I slot print_timing: id  0 | task 0 | n_decoded =   4144, tg =  12.69 t/s, tg_3s =  12.05 t/s
6.14.727.625 I slot print_timing: id  0 | task 0 | n_decoded =   4180, tg =  12.68 t/s, tg_3s =  11.87 t/s
6.17.798.800 I slot print_timing: id  0 | task 0 | n_decoded =   4218, tg =  12.68 t/s, tg_3s =  12.37 t/s
6.20.856.354 I slot print_timing: id  0 | task 0 | n_decoded =   4256, tg =  12.67 t/s, tg_3s =  12.43 t/s
6.23.922.881 I slot print_timing: id  0 | task 0 | n_decoded =   4294, tg =  12.67 t/s, tg_3s =  12.39 t/s
6.26.929.535 I slot print_timing: id  0 | task 0 | n_decoded =   4331, tg =  12.67 t/s, tg_3s =  12.31 t/s
6.29.946.912 I slot print_timing: id  0 | task 0 | n_decoded =   4368, tg =  12.66 t/s, tg_3s =  12.26 t/s
6.32.976.917 I slot print_timing: id  0 | task 0 | n_decoded =   4405, tg =  12.66 t/s, tg_3s =  12.21 t/s
6.36.035.841 I slot print_timing: id  0 | task 0 | n_decoded =   4442, tg =  12.65 t/s, tg_3s =  12.10 t/s
6.39.095.748 I slot print_timing: id  0 | task 0 | n_decoded =   4479, tg =  12.65 t/s, tg_3s =  12.09 t/s
6.42.172.908 I slot print_timing: id  0 | task 0 | n_decoded =   4516, tg =  12.64 t/s, tg_3s =  12.02 t/s
6.45.196.060 I slot print_timing: id  0 | task 0 | n_decoded =   4552, tg =  12.64 t/s, tg_3s =  11.91 t/s
6.48.269.590 I slot print_timing: id  0 | task 0 | n_decoded =   4588, tg =  12.63 t/s, tg_3s =  11.71 t/s
6.51.334.403 I slot print_timing: id  0 | task 0 | n_decoded =   4626, tg =  12.63 t/s, tg_3s =  12.40 t/s
6.54.410.759 I slot print_timing: id  0 | task 0 | n_decoded =   4664, tg =  12.63 t/s, tg_3s =  12.35 t/s
6.57.412.470 I slot print_timing: id  0 | task 0 | n_decoded =   4701, tg =  12.62 t/s, tg_3s =  12.33 t/s
7.00.424.732 I slot print_timing: id  0 | task 0 | n_decoded =   4738, tg =  12.62 t/s, tg_3s =  12.28 t/s
7.03.431.519 I slot print_timing: id  0 | task 0 | n_decoded =   4775, tg =  12.62 t/s, tg_3s =  12.31 t/s
7.06.457.939 I slot print_timing: id  0 | task 0 | n_decoded =   4812, tg =  12.62 t/s, tg_3s =  12.23 t/s
7.09.514.110 I slot print_timing: id  0 | task 0 | n_decoded =   4849, tg =  12.61 t/s, tg_3s =  12.11 t/s
7.12.578.332 I slot print_timing: id  0 | task 0 | n_decoded =   4886, tg =  12.61 t/s, tg_3s =  12.07 t/s
7.15.606.924 I slot print_timing: id  0 | task 0 | n_decoded =   4922, tg =  12.60 t/s, tg_3s =  11.89 t/s
7.18.661.909 I slot print_timing: id  0 | task 0 | n_decoded =   4958, tg =  12.60 t/s, tg_3s =  11.78 t/s
7.21.665.964 I slot print_timing: id  0 | task 0 | n_decoded =   4993, tg =  12.59 t/s, tg_3s =  11.65 t/s
7.24.682.783 I slot print_timing: id  0 | task 0 | n_decoded =   5030, tg =  12.59 t/s, tg_3s =  12.26 t/s
7.27.686.353 I slot print_timing: id  0 | task 0 | n_decoded =   5067, tg =  12.58 t/s, tg_3s =  12.32 t/s
7.29.643.432 I slot print_timing: id  0 | task 0 | prompt eval time =    3153.97 ms /   171 tokens (   18.44 ms per token,    54.22 tokens per second)
7.29.643.437 I slot print_timing: id  0 | task 0 |        eval time =  404621.74 ms /  5091 tokens (   79.48 ms per token,    12.58 tokens per second)
7.29.643.437 I slot print_timing: id  0 | task 0 |       total time =  407775.71 ms /  5262 tokens
7.29.643.438 I slot print_timing: id  0 | task 0 |    graphs reused =       5070
7.29.643.620 I slot      release: id  0 | task 0 | stop processing: n_tokens = 5261, truncated = 0

Metadata

Metadata

Assignees

Labels

hardware: hackintoshOpenCore or other non-Apple macOS hardwarestatus: needs reproductionPlausible report that needs independent or repeated reproductiontype: performancePerformance regression, optimization, or benchmark finding

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions