Support for asynchronous requests for watsonx.ai chat

[pawelknes]

Added support for asynchronous requests in the WMLInferenceEngineChat. The default concurrency limit is set to be the same as in the case of the WMLInferenceEngineGeneration: 10.

Small performance test (for the dataset at the bottom, averaged for 3 runs each):

• old version: 31.2 seconds
• new version (concurrency_limit=10): 6.4 seconds

dataset = load_dataset(
    card="cards.wnli",
    template_card_index=0,
    format="formats.chat_api",
    loader_limit=21,
)["test"]

[yoavkatz]

Looks good. I've merged to check it in my branch of testing. The unit tests are failing but might not be related. @elronbandel and @eladven are looking at it.

[pawelknes]

@yoavkatz one error is related to the obsolete version of ibm-watsonx-ai(I've pushed a commit with the upgrade), the other is related to the test_lite_llm_inference_engine (it happened previously as well)

[pawelknes]

@yoavkatz one error is related to the obsolete version of ibm-watsonx-ai(I've pushed a commit with the upgrade), the other is related to the test_lite_llm_inference_engine (it happened previously as well)

I see the catalog consistency test failing as well but it seems to be an issue with HF

[pawelknes]

@yoavkatz
I see that the inference test keeps failing for LiteLLM tests with:
litellm.exceptions.APIConnectionError: litellm.APIConnectionError: WatsonxException - Client error '400 Bad Request' for url
I saw the same error already happening before, so I don't think it's related to the changes in this PR