Eval bug: --fit-target projection miscalculation with MTP on ROCm

[eaman]

Name and Version

./llama-server --version
version: 746 (da296d6)
built with GNU 15.3.0 for Linux x86_64

Operating systems

Linux

GGML backends

HIP

Hardware

CPU : 1th Gen Intel(R) Core(TM) i5-11400F
GPU: AMD Radeon RX 6800

Models

https://huggingface.co/GianniDPC/Qwen3.6-27B-IQ4_XS-pure-with-MTP-GGUF
qwen3.6-27b-IQ4_XS-pure-with-MTP-IQ4.gguf

Problem description & steps to reproduce

When allocating ctx size with --fit-target 60 ROCm takes way much more space than vulkan for the same model:

ROCm
rocm with mtp: 1940 ctx

I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
I common_memory_breakdown_print: | - ROCm0 (RX 6800) | 16368 = 15708 + (13826 = 13244 + 544 + 37) + -13166 |
I common_memory_breakdown_print: | - Host | 650 = 644 + 0 + 6 |
I common_params_fit_impl: context size reduced from 262144 to 19456 -> need 6695 MiB less memory in total

rocm no mtp: 9190 ctx

I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
I common_memory_breakdown_print: | - ROCm0 (RX 6800) | 16368 = 16010 + (13527 = 13244 + 245 + 37) + -13169 |
I common_memory_breakdown_print: | - Host | 650 = 644 + 0 + 6 |
I common_params_fit_impl: context size reduced from 262144 to 91904 -> need 4696 MiB less memory in t

============ END ROCm

Vulkan:
with mtp: 6912 ctx

I common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
I common_memory_breakdown_print: | - Vulkan0 (RX 6800 (RADV NAVI21)) | 16368 = 15800 + (13828 = 13244 + 544 + 38) + -13261 |
I common_memory_breakdown_print: | - Host | 650 = 644 + 0 + 6 |
I common_params_fit_impl: context size reduced from 262144 to 69120 -> need 4571 MiB less memory in total

without mtp: 11008

common_memory_breakdown_print: | memory breakdown [MiB] | total free self model context compute unaccounted |
I common_memory_breakdown_print: | - Vulkan0 (RX 6800 (RADV NAVI21)) | 16368 = 16092 + (13524 = 13244 + 245 + 34) + -13249 |
I common_memory_breakdown_print: | - Host | 650 = 644 + 0 + 6 |
I common_params_fit_impl: context size reduced from 262144 to 110080 -> need 3600 MiB less memory in total

================ END Vulkan

So it looks like:

MTP cost:
ROCm 9190 - 1940 = 7250 ctx /
vulkan 11008 - 6912 = 4096 ctx /

Compute memory and --fit-target margin are the same yet llama.cpp's HIP is overestimating required GPU memory by roughly 800 MiB when I use MTP.

---

Launch script:

llama-server --device rocm0
-m /home/eaman/lm/models/gianni/qwen3.6-27b-IQ4_XS-pure-with-MTP-IQ4.gguf
--host 0.0.0.0 -fa on --no-mmap --jinja --chat-template-file /home/eaman/lm/models/chat_template.jinja --no-log-timestamps
-ctk q5_1 -ctv q5_1
--temp 0.5 --top-k 32 --top-p 0.95 --min-p 0.05
--presence-penalty 0.0 --repeat-penalty 1.0
-b 1024 -ub 128
--fit-target 60
--spec-type draft-mtp,ngram-mod --spec-draft-p-min 0.82 --spec-draft-n-max 2
--cache-type-k-draft q4_0 --cache-type-v-draft q4_0
--spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 8 --spec-ngram-mod-n-max 32
--ctx-checkpoints 128 --cache-ram 6000 -np 1 -ngl 99 -lv 4 --no-warmup \

First Bad Commit

No response

Relevant log output

Logs