Skip to content

Add kv_cache size to pipeline generation log.#3989

Open
rasapala wants to merge 2 commits intomainfrom
kvcache_size_log
Open

Add kv_cache size to pipeline generation log.#3989
rasapala wants to merge 2 commits intomainfrom
kvcache_size_log

Conversation

@rasapala
Copy link
Collaborator

@rasapala rasapala commented Feb 18, 2026

🛠 Summary

JIRA CVS-181044
Depends on https://github.com/openvinotoolkit/openvino.genai/pull/3352/changes and ovms genai update.
Output:
Cache usage 0.5% of 1022.4 MB; for static 1GB
Cache usage 100.0% of 5.2 MB; for dynamic cache_size = 0

For: "usage":{"prompt_tokens":25,"completion_tokens":30,"total_tokens":55}}

🧪 Checklist

  • Unit tests added.
  • The documentation updated.
  • Change follows security best practices.
    ``

@rasapala rasapala requested review from dtrawins and mzegla February 18, 2026 15:44
Copy link

@popovaan popovaan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Copy link
Collaborator

@mzegla mzegla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does not seem to make sense to report 100% of X in every log for dynamic cache size.
It's a minor, but can we drop showing percentage for dynamic cache size and only report size in MB/GB etc. ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments