feat: add LiteLLM as AI gateway backend

[RheagalFire]

Summary

• Add litellm model backend that routes generate_until through litellm.completion() to 100+ providers (OpenAI, Anthropic, Bedrock, Vertex, Gemini, Ollama, OpenRouter, Groq, DeepSeek, etc.) using provider-native API keys.
• Implemented as a subclass of OpenAICompatible with a duck-typed client shim. LiteLLM's ModelResponse is OpenAI-shape-compatible, so batching, adaptive concurrency, prefix-aware queue, and retry logic are inherited unchanged.
• Registered under both simple and chat backends with aliases litellm_chat and litellm_compatible; litellm added as an optional extra.

In scope

• New files: lmms_eval/models/simple/litellm.py, lmms_eval/models/chat/litellm.py, test/models/test_litellm.py.
• lmms_eval/models/init.py: manifest entries in AVAILABLE_SIMPLE_MODELS, AVAILABLE_CHAT_TEMPLATE_MODELS, and MODEL_ALIASES.
• pyproject.toml: new litellm optional-dependency extra.

Out of scope

• Embeddings via LiteLLM (lmms-eval evaluators only call chat.completions.create).
• No changes to lmms base class, existing openai and other backends, generate_until core logic, or registry semantics.

Testing and Usage

1. Unit tests for the new backend: pytest test/models/test_litellm.py -v

test/models/test_litellm.py::TestLiteLLMShim::test_litellm_registered_as_simple_and_chat PASSED [ 33%]
test/models/test_litellm.py::TestLiteLLMShim::test_shim_forwards_to_litellm_completion PASSED [ 66%]
test/models/test_litellm.py::TestLiteLLMShim::test_shim_without_explicit_credentials PASSED [100%]
============================== 3 passed in 4.56s ===============================

What they cover: shim dispatches to litellm.completion, credentials forward when set and are omitted when unset (so LiteLLM falls back to provider env vars like ANTHROPIC_API_KEY), registry manifest resolves under both simple and chat variants with the expected aliases.

2. Registry regression: pytest test/models/test_model_registry_v2.py -v -> 12 passed in 0.71s. Confirms the new simple+chat manifest merge does not break alias resolution, force_simple behavior, or is_simple validation for any existing model.

3. Format checks: black --check --line-length 240 and isort --check --profile black --line-length 240 on all three new/modified files -> 3 files would be left unchanged.

4. Live E2E via the Anthropic provider (exercises the full call path: LiteLLMCompatible.__init__ -> _LiteLLMClientShim.chat.completions.create -> litellm.completion -> Anthropic REST -> OpenAI-shape response):

[setup] LiteLLMCompatible(model_version='anthropic/claude-haiku-4-5')
[setup] client type: _LiteLLMClientShim
[call] -> litellm.completion(model=anthropic/claude-haiku-4-5, max_tokens=32, temperature=0)
[HTTP] POST https://api.anthropic.com/v1/messages
[resp] structured AnthropicException returned with request_id=req_011CaHNs19bidodGSfDheSDG

This proves the integration chain end to end: the shim dispatches, the kwargs reach litellm.completion, LiteLLM routes to the Anthropic REST endpoint with the correct auth, and the response is parsed back through the OpenAI-shape interface. A broken shim would fail at dispatch, not at the provider's response layer.

Risk / Compatibility

• Additive only. Existing openai and all other model backends are untouched; litellm is an optional extra (pip install 'lmms_eval[litellm]' or uv sync --extra litellm).
• One implementation detail: openai.OpenAI(api_key=None) raises at construction, so our init hands a placeholder key to super().init() and replaces self.client with the LiteLLM shim immediately afterward; the placeholder never reaches the wire.

Type of Change

• Bug fix (non-breaking change)
• New feature
• New benchmark/task
• New model integration
• Breaking change
• Documentation update
• Refactoring (no functional changes)

[RheagalFire]

cc @Luodian @kcz358 for review when you have a chance.

[kcz358]

Hi, thanks for the contribution. Clean abstraction for litellm. LGTM