feat: add azure openai inference

[leseb]

feat: add azure openai inference

New inference provider.

Relates to: RHAIENG-744
Signed-off-by: Sébastien Han seb@redhat.com

@coderabbitai ignore

[coderabbitai]

Note

Reviews paused

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Walkthrough

Bumps llama-stack to 0.2.22 across configs, adds a conditional Azure remote inference provider to build/run configurations and docs, and updates container dependencies by replacing openai with litellm. Pre-commit hook and build scripts align with the new version; no API signatures changed.

Changes

Cohort / File(s)	Summary
Version bump: llama-stack 0.2.21 → 0.2.22 .pre-commit-config.yaml, distribution/Containerfile, distribution/Containerfile.in, distribution/build.py	Update pinned/expected llama-stack version to 0.2.22 in pre-commit, container build, and build script requirements.
Azure inference provider (config + docs) distribution/build.yaml, distribution/run.yaml, distribution/README.md	Add remote Azure inference provider: static entry in build.yaml; conditional provider in run.yaml using env vars; document “inference
Container dependency change distribution/Containerfile	Replace Python dependency openai with litellm in pip install list; no other container changes.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant U as User
  participant R as Runner (distribution/run.yaml)
  participant PR as Provider Registry
  participant AZ as Azure Provider (remote::azure)
  participant Other as Other Inference Providers

  rect rgba(200,220,255,0.25)
    note over R,PR: Provider registration at startup
    R->>PR: Register providers from config
    alt env.AZURE_API_KEY is set
      note right of PR: Conditionally include Azure
      PR->>PR: Add provider_id "azure"
    else env.AZURE_API_KEY is empty
      PR->>PR: Skip Azure provider
    end
  end

  U->>R: Submit inference request
  R->>PR: Select provider (routing rules)
  alt Azure selected
    PR->>AZ: Forward request with api_key/base/version/type
    AZ-->>PR: Response
  else Other provider selected
    PR->>Other: Forward request
    Other-->>PR: Response
  end
  PR-->>R: Return response
  R-->>U: Deliver result

  note over R,A Z: New/changed: conditional Azure provider integration

Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

Ears up, I flip through YAML skies,
Azure clouds wink—surprise, surprise!
Llama trots to .22, so spry,
litellm packs our carrot pie.
Hooks are tidy, builds run true—
I thump in joy: “Ship it, woo!” 🥕✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changes. Docstring coverage check skipped.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title Check	✅ Passed	The title "feat: add azure openai inference" is a concise, single-sentence summary that correctly highlights the primary change in the PR (adding an Azure OpenAI inference provider); the bump to llama-stack is a secondary dependency update and does not make the title misleading.

✨ Finishing touches

• 📝 Generate Docstrings

🧪 Generate unit tests

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[leseb]

@coderabbitai ignore

[Elbehery]

@leseb the downstream release does not include azure

Downstream release was cut at this MR

[coderabbitai]

✅ Actions performed

Reviews paused.

[leseb]

needs #39

[Elbehery]

pre-commit fails on

Error executing command: Command 'llama stack build --config distribution/build.yaml --print-deps-only' returned non-zero exit status 1.
Command output: # Dependencies for distribution/build.yaml
WARNING  2025-09-18 09:21:29,362 llama_stack.core.distribution:149 core: Failed to import module prompts: No module named                             
         'llama_stack.providers.registry.prompts'                                                                                                     

Command stderr: Traceback (most recent call last):
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/bin/llama", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/cli/llama.py", line 53, in main
    parser.run(args)
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/cli/llama.py", line 47, in run
    args.func(args)
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/cli/stack/build.py", line 100, in _run_stack_build_command
    return run_stack_build_command(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/cli/stack/_build.py", line 232, in run_stack_build_command
    normal_deps, special_deps, external_provider_dependencies = get_provider_dependencies(build_config)
                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/core/build.py", line 65, in get_provider_dependencies
    raise ValueError(f"Provider `{provider}` is not available for API `{api_str}`")
ValueError: Provider `provider_type='remote::azure' module=None` is not available for API `inference`

[leseb]

pre-commit fails on

Error executing command: Command 'llama stack build --config distribution/build.yaml --print-deps-only' returned non-zero exit status 1.
Command output: # Dependencies for distribution/build.yaml
WARNING  2025-09-18 09:21:29,362 llama_stack.core.distribution:149 core: Failed to import module prompts: No module named                             
         'llama_stack.providers.registry.prompts'                                                                                                     

Command stderr: Traceback (most recent call last):
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/bin/llama", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/cli/llama.py", line 53, in main
    parser.run(args)
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/cli/llama.py", line 47, in run
    args.func(args)
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/cli/stack/build.py", line 100, in _run_stack_build_command
    return run_stack_build_command(args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/cli/stack/_build.py", line 232, in run_stack_build_command
    normal_deps, special_deps, external_provider_dependencies = get_provider_dependencies(build_config)
                                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/.cache/pre-commit/repoiv_b5t3a/py_env-python3.12/lib/python3.12/site-packages/llama_stack/core/build.py", line 65, in get_provider_dependencies
    raise ValueError(f"Provider `{provider}` is not available for API `{api_str}`")
ValueError: Provider `provider_type='remote::azure' module=None` is not available for API `inference`

As expected since this branch does not run on 0.2.22, hence the wait for

[leseb]

@nathan-weinberg @Elbehery ready!

[Elbehery]

LGTM