[NPUW] 1st token generation profiling

[dylanneve1]

Details:

• Adds LLM-level execution profiling to LLMInferRequest for 1st token generation analysis, enabled with OPENVINO_NPUW_PROF=1. Breaks down prefill into prepare (fill inputs, clear kvcache...), infer, and LM head phases, and generate into copy kvcache, initialize inputs, prepare, infer and post phases. Metrics are printed in chronological execution order.

Tickets:

• EISW-202385

[dmatveev]

Now looking at the profiling structure, what we get is:

LLM/execution:
  1/prefill:1.prepare
  1/prefill:1a.fill_inputs
  1/prefill:1b.clear_kvcache
  1/prefill:1c.select_gen_req
  1/prefill:1d.apply_lora
  1/prefill:2.infer
  1/prefill:2a.npu_infer
  1/prefill:3.lm_head
  N/generate:1.copy_kvcache
  N/generate:2.init_inputs
  N/generate:3.prepare
  N/generate:4.infer
  N/generate:5.post

Looking at the actual timings, I think

1. prefill:1.prepare is way to detailed for the time spent - we can leave it as a single entry? Appying LoRA adapters may be the only costly thing
2. npu_infer may be misleading as, technically, the underlying model can run on any IP.. Also there's a subtle delta between prefill:2. and prefill:2.a - should we see it on the longer contexts?
3. lm_head is shown in prefill stage only but in fact it runs in both - I think it should be possible to have two regions for the same model?
4. do you have kvcache update instrumented in 2nd token? Or is it asynchronous?

[dylanneve1]

Thank you, I believe these should be addressed now. Current profiling format is bellow

LLM/execution:
  1/prefill:1.prepare_for_new_conversation[ ... ]
  1/prefill:2.apply_lora[ ... ]
  1/prefill:3.infer   [ ... ]
  1/prefill:4.lm_head [ ... ]
  N/generate:1.prepare[ ... ]
  N/generate:2.infer  [ ... ]
  N/generate:3.update_kvcache[ ... ]
  N/generate:4.lm_head[ ... ]

The fill_in/copy_n you mentioned are now included in top level profiling as well, moved to a top level infer profiling in infer_prefill instead of just profiling m_prefill_request->infer().

[Copilot]

Pull request overview

Adds request-level execution profiling for NPUW LLM inference to help analyze first-token generation behavior when OPENVINO_NPUW_PROF=1 is enabled.

Changes:

• Introduces an ov::npuw::perf::Profile member on LLMInferRequest for LLM-level timing metrics.
• Instruments infer_prefill() and infer_generate() with phase-level timing buckets (prepare/apply_lora/infer/lm_head, etc.).
• Configures profiling output area/tagging and enables reporting based on ov::npuw::profiling_enabled().

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
src/plugins/intel_npu/src/plugin/npuw/llm_infer_request.hpp	Adds perf.hpp include and a Profile member to store LLM-level timing metrics.
src/plugins/intel_npu/src/plugin/npuw/llm_infer_request.cpp	Enables the profile in the ctor and wraps key prefill/generate phases with profiling timers and tagged metrics.

---

You can also share your feedback on Copilot code review. Take the survey.

[dmatveev]

Order may be important here

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

---

You can also share your feedback on Copilot code review. Take the survey.