To get Performance Metrics in Client Response

[lililolo0927]

I am currently using the API server with the following command

PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True python3 ktransformers/server/main.py \
--gguf_path /models/gguf/DeepSeek-V3-Q4_K_M \
--model_path deepseek-ai/DeepSeek-V3 \
--model_name DeepSeek-V3 \
--cpu_infer 17 \
--cache_q4 true \
--top_p 0.95 \
--temperature 0.1 \
--use_cuda_graph \
--host 0.0.0.0 \
--port 10002

in Ktransformers, but I would like to receive the following performance information, which appears on the server side, as part of the response to the API request on the client side:

/workspace/ktransformers/ktransformers/server/backend/interfaces/transformers.py[333]: args.max_new_tokens: 2000, cache_lens: 8192, seq_length: 1652 
/workspace/ktransformers/ktransformers/server/backend/interfaces/transformers.py[338]: max_new_tokens: 199 
/workspace/ktransformers/ktransformers/server/backend/base.py's Performance(T/s): prefill 13.242479595553231, decode 1.1463979978574244. Time(s): tokenize 0.007807254791259766, prefill 122.93770122528076, decode 1.7445948123931885

Which code should I modify to achieve this? If there is a more efficient method, please suggest it.
Thank u in advance!

[vibe-Chen]

If you wanna to send these info to client side... how will you proceed them?
In general, you send a HTTP request to server via client, and server respond with data. If client can't proceed those "extra" data, most of them will just throw those data away.
You may want to modify BOTH server AND client. Server send extra data to client, and client MUST TO BE ABLE to process them.
I have no good idea about how or where to modify the server, but as I know, some of the clients already have statistics, apart from server-side information, though with certain limitations.
I won't suggest Chatbox, as it's close-source. You may try open-webui, though that would need a bit of configurations.

[lililolo0927]

open-webui was good idea! Thanks!