Why does finer granularity in quantization result in more overhead?

[lsjlsj5846]

Hello,

Your paper says "finer granularity in quantization results in more overhead but less information loss," but I do not understand why.
Could you explain why per-tensor quantization results in faster inference than per-token quantization?

Thank you.