Add function to create usm host tensor

[ahnyoung-paul]

After the Whisper model's commit (07dfba94a36f88f0614b8bb61ebb2d9b8ef6324c) is merged, beam_idx and encoder_hidden_states are created as ov::Tensor in the Whisper pipeline. However, this creation of ov::Tensor in GenAI may affect performance. In the GPU plugin, the tensor obtained from infer_request is a host tensor that uses usm_host memory, while the newly created tensor is an ov::allocated_tensor.

When ov::allocatedTensor is used as an input for prepare_input, it is neither a remote tensor nor a usm_host tensor. Consequently, the GPU plugin internally creates a usm_device tensor and performs unnecessary copying of values, leading to degraded performance.

To fix this issue, use the create_host_tensor method instead of creating ov::Tensor directly.

This is my issue ticket: CVS-162818

[Wovchena]

When I was doing a similar thing for VLM m_request.get_compiled_model().get_context() didn't bring any performance. But reusing the context did. My guess was that m_request.get_compiled_model().get_context() takes too much time.

Did you verify that the issue is gone with you patch?

[ilya-lavrenov]

BTW, get_context() is not available for CPU

[andrei-kochin]

@ahnyoung-paul Did you verify that the issue is gone with you patch?

[Wovchena]

Exception handling is known to be costly in C++. With that try catch you need to also verify that CPU performance didn't degrade

[ahnyoung-paul]

Exception handling is known to be costly in C++. With that try catch you need to also verify that CPU performance didn't degrade

Verified no found regression for CPU, please find detail from the ticket. thanks.

[ahnyoung-paul]

@ahnyoung-paul Did you verify that the issue is gone with you patch?

This issue is not caused by a single factor but by multiple issues leading to performance degradation. By applying this PR to the Whisper model's commit (07dfba9), most of the issues were resolved, but another performance degradation was found in the latest commit. This PR reduces unnecessary enqueue memory creation and copying, including enqueue memcpy, thereby reducing device time and somewhat improving latency. However, further improvements require additional analysis from the GenAI team. Detailed information has been left in the ticket (CVS-162818), thanks.