feat: add watsonx inference provider and bump lmeval

[leseb]

8b82642 feat: add watsonx inference provider
ffce08d chore: bump lmeval

commit 8b82642
Author: Sébastien Han seb@redhat.com
Date: Fri Sep 12 17:10:05 2025 +0200

feat: add watsonx inference provider

New provider, env variable WATSONX_API_KEY and WATSONX_PROJECT_ID are
expected to be populated to enable the provider.

Relates to: RHAIENG-1026
Signed-off-by: Sébastien Han <seb@redhat.com>

commit ffce08d
Author: Sébastien Han seb@redhat.com
Date: Fri Sep 12 17:22:21 2025 +0200

chore: bump lmeval

Fix auth token propagation in the safety provider.

Relates to: RHOAIENG-34050
Signed-off-by: Sébastien Han <seb@redhat.com>

@coderabbitai ignore

Summary by CodeRabbit

• New Features

  • Added IBM watsonx as a remote inference provider, enabling integration with Watsonx services.
  • Provider configuration supports environment variables for base URL, API key, and project ID.
  • Included in default build and runtime configurations for easy enablement.

• Chores

  • Container image now installs the IBM watsonx AI Python client during build, ensuring compatibility with the new provider.

[leseb]

@coderabbitai ignore

[coderabbitai]

Note

Reviews paused

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Caution

Failed to replace (edit) comment. This is likely due to insufficient permissions or the comment being deleted.

Error details

{"name":"HttpError","status":404,"request":{"method":"PATCH","url":"https://api.github.com/repos/opendatahub-io/llama-stack-distribution/issues/comments/3285668678","headers":{"accept":"application/vnd.github.v3+json","user-agent":"octokit.js/0.0.0-development octokit-core.js/7.0.3 Node.js/24","authorization":"token [REDACTED]","content-type":"application/json; charset=utf-8"},"body":{"body":"<!-- This is an auto-generated comment: summarize by coderabbit.ai -->\n<!-- walkthrough_start -->\n\n## Walkthrough\nAdds IBM WatsonX AI client to the container build and registers a new remote WatsonX inference provider in build and run configurations, including URL, API key, and project ID settings sourced from environment variables.\n\n## Changes\n| Cohort / File(s) | Summary |\n|---|---|\n| **Container image dependencies**<br>`distribution/Containerfile` | Added `ibm_watsonx_ai` to pip install list for image build. |\n| **Inference provider configuration**<br>`distribution/build.yaml`, `distribution/run.yaml` | Introduced `remote::watsonx` provider type and a `watsonx` provider entry with `url`, `api_key`, and `project_id` (env-driven). |\n\n## Sequence Diagram(s)\n```mermaid\nsequenceDiagram\n  autonumber\n  actor User\n  participant App as Inference Orchestrator\n  participant Provider as WatsonX Provider Adapter\n  participant WatsonX as IBM WatsonX API\n\n  User->>App: Submit inference request\n  App->>Provider: Route to provider_type=remote::watsonx\n  Note over Provider: Load config from env<br/>WATSONX_BASE_URL, WATSONX_API_KEY, WATSONX_PROJECT_ID\n  Provider->>WatsonX: HTTPS request with API key, project ID, payload\n  WatsonX-->>Provider: Inference response / error\n  Provider-->>App: Normalize and return result\n  App-->>User: Deliver inference output\n```\n\n## Estimated code review effort\n🎯 2 (Simple) | ⏱️ ~10 minutes\n\n\n\n\n\n\n## Poem\n> I packed my kit with WatsonX light,  \n> a URL, a key—set for flight.  \n> New routes in YAML, neat and shy,  \n> remote calls whisking answers by.  \n> Thump-thump goes my rabbit heart—  \n> configs aligned, we’re ready to start! 🐇✨\n\n<!-- walkthrough_end -->\n\n\n"},"request":{"retryCount":1}},"response":{"url":"https://api.github.com/repos/opendatahub-io/llama-stack-distribution/issues/comments/3285668678","status":404,"headers":{"access-control-allow-origin":"*","access-control-expose-headers":"ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset","content-encoding":"gzip","content-security-policy":"default-src 'none'","content-type":"application/json; charset=utf-8","date":"Fri, 12 Sep 2025 15:19:33 GMT","referrer-policy":"origin-when-cross-origin, strict-origin-when-cross-origin","server":"github.com","strict-transport-security":"max-age=31536000; includeSubdomains; preload","transfer-encoding":"chunked","vary":"Accept-Encoding, Accept, X-Requested-With","x-accepted-github-permissions":"issues=write; pull_requests=write","x-content-type-options":"nosniff","x-frame-options":"deny","x-github-api-version-selected":"2022-11-28","x-github-media-type":"github.v3; format=json","x-github-request-id":"1062:315E29:BCD657:2A8D417:68C43A05","x-ratelimit-limit":"12500","x-ratelimit-remaining":"12083","x-ratelimit-reset":"1757693018","x-ratelimit-resource":"core","x-ratelimit-used":"417","x-xss-protection":"0"},"data":{"message":"Not Found","documentation_url":"https://docs.github.com/rest/issues/comments#update-an-issue-comment","status":"404"}}}

Pre-merge checks (3 passed)

✅ Passed checks (3 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title accurately highlights the primary change—adding a watsonx inference provider—which matches the new remote::watsonx entries in distribution/run.yaml and distribution/build.yaml and the IBM client added to the Containerfile; this makes the title relevant and informative. However, the title also mentions "bump lmeval", which is not reflected in the provided file summaries, so that part may be missing or inaccurate. Overall the title is relevant to the main change but includes an unsupported claim about lmeval.

✨ Finishing touches

• 📝 Generate Docstrings

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (2)

distribution/Containerfile (1)

18-18: Use canonical PyPI name and (ideally) pin the version for reproducible builds.

pip normalizes names, but the published package is “ibm-watsonx-ai”. Consider also pinning to a known-good version to avoid surprise breakages.

Apply:

-    ibm_watsonx_ai \
+    ibm-watsonx-ai \

Optionally:

-    ibm-watsonx-ai \
+    ibm-watsonx-ai==X.Y.Z \

If you want me to propose a tested version matrix for UBI9 + Py3.12, say the word.

distribution/run.yaml (1)

39-44: Wire a model to the new provider so it’s usable out-of-the-box.

No model currently targets provider_id: watsonx. Add a model entry controlled by an env var.

Add under models: (outside this hunk):

- metadata: {}
  model_id: ${env.WATSONX_MODEL_ID:=watsonx-llm}
  provider_id: watsonx
  provider_model_id: ${env.WATSONX_PROVIDER_MODEL_ID:=}
  model_type: llm

I can align names with your existing conventions if you share the desired default model(s).

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 4e88d5d and 8b82642.

📒 Files selected for processing (3)

• distribution/Containerfile (1 hunks)
• distribution/build.yaml (1 hunks)
• distribution/run.yaml (1 hunks)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: build-test-push (linux/amd64)

🔇 Additional comments (1)

distribution/build.yaml (1)

9-9: Verify remote::watsonx exists in the bundled llama-stack (0.2.21) or add a module mapping.

File: distribution/build.yaml (line 9) — provider_type: remote::watsonx.

Automated check failed in the verification environment (ModuleNotFoundError: No module named 'llama_stack'). Run this locally to confirm the provider registry:

python - <<'PY'
import importlib, pkgutil
m = importlib.import_module("llama_stack.providers")
print([name for _, name, _ in pkgutil.walk_packages(m.__path__, m.__name__ + ".") if name.endswith(".watsonx")])
PY

If remote::watsonx is absent in the pinned 0.2.21, either bump llama-stack to a version that includes it or add a module: <watsonx provider package>==<version> entry alongside this provider_type in distribution/build.yaml.

[coderabbitai]

✅ Actions performed

Reviews paused.

[leseb]

draft since downstream hasn't merged yet

[leseb]

Ready!