File tree Expand file tree Collapse file tree 1 file changed +22
-0
lines changed
Expand file tree Collapse file tree 1 file changed +22
-0
lines changed Original file line number Diff line number Diff line change @@ -54,3 +54,25 @@ AFFINITY: CORE
5454EXECUTION_DEVICES:
5555CPU: Intel(R) Xeon(R) Platinum 8468
5656```
57+
58+ When Speculative Decoding ot Prompt Lookup pipeline is executed, performance metrics will be also printed.
59+
60+ ``` sh title="Output:"
61+ ===============================
62+ Total duration, sec: 26.6217
63+ Draft model duration, sec: 1.60329
64+ Main model duration, sec: 25.0184
65+ Draft model duration, %: 6.02248
66+ Main model duration, %: 93.9775
67+ AVG acceptance rate, %: 21.6809
68+ ===============================
69+ REQUEST_ID: 0
70+ Main model iterations: 47
71+ Token per sec: 3.75633
72+ AVG acceptance rate, %: 21.6809
73+ Accepted tokens by draft model: 51
74+ Generated tokens: 100
75+ Accepted token rate, %: 51
76+ ===============================
77+ Request_id: 0 ||| 40 0 40 20 0 0 40 40 0 20 20 20 0 40 0 0 20 80 0 80 20 0 0 0 40 80 0 40 60 40 80 0 0 0 0 40 20 20 0 40 20 40 0 20 0 0 0
78+ ```
You can’t perform that action at this time.
0 commit comments