Skip to content

feat(001): implement deploy-time external providers L1#240

Draft
rhuss wants to merge 6 commits into
ogx-ai:legacyfrom
rhuss:spec/001/implement
Draft

feat(001): implement deploy-time external providers L1#240
rhuss wants to merge 6 commits into
ogx-ai:legacyfrom
rhuss:spec/001/implement

Conversation

@rhuss

@rhuss rhuss commented Jan 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

This PR implements the core functionality for external provider injection at deployment time, enabling custom LlamaStack providers via container images.

Spec Reference: specs/001-deploy-time-providers-l1/spec.md

CRD Extensions

  • ExternalProvidersSpec organized by API type (inference, safety, agents, etc.)
  • ExternalProviderRef with providerId, image, imagePullPolicy, config
  • ExternalProviderStatus for phase-based tracking (Pending, Installing, Ready, Failed)

New Packages

Provider Package (pkg/provider/)

  • ProviderMetadata struct matching lls-provider-spec.yaml schema
  • LoadProviderMetadata and ParseProviderMetadata functions
  • ValidateMetadata with comprehensive validation rules
  • API name normalization between CRD and config.yaml formats

Deploy Package Extensions (pkg/deploy/)

  • GenerateInitContainers for provider installation
  • GenerateExtractConfigInitContainer for base config extraction
  • Volume helpers for shared emptyDir configuration
  • RunYamlConfig struct and MergeExternalProviders function
  • GenerateConfig orchestrating the full merge process

Binary (cmd/generate-config/)

  • CLI tool for config generation init container
  • Reads provider metadata and merges into base config.yaml

Controller Integration

  • configureExternalProviders in resource_helper.go
  • Status tracking functions in status.go
  • PYTHONPATH and config path updates for main container

Dockerfile Updates

  • Builds and includes generate-config binary at /generate-config

Example Usage

apiVersion: llama.meta.com/v1alpha1
kind: LlamaStackDistribution
metadata:
  name: my-llsd
spec:
  server:
    distribution:
      name: ollama
    externalProviders:
      volumeSizeLimit: "2Gi"
      inference:
        - providerId: custom-vllm
          image: registry.example.com/custom-vllm-provider:v1.0
          config:
            url: https://my-vllm-endpoint.example.com
      safety:
        - providerId: custom-safety
          image: registry.example.com/safety-provider:v1.0

Test plan

  • Unit tests pass for pkg/provider/ package
  • Unit tests pass for pkg/deploy/ package (initcontainer, runyaml)
  • Integration test with single external provider
  • Integration test with multiple providers across different API types
  • Verify PYTHONPATH is correctly updated in main container
  • Verify config.yaml is correctly merged with external provider entries
  • Verify status tracking reflects provider installation phases

Assisted-By: 🤖 Claude Code

rhuss added 6 commits January 29, 2026 18:10
Align spec files with llama-stack upstream refactoring:

Naming changes:
- run.yaml -> config.yaml (157 references)
- --run-yaml -> --config CLI flags
- image_name -> distro_name (deprecated field)

Architecture simplification:
- Remove extra-providers.yaml and merge step entirely
- Use LlamaStack's native `module:` field in provider entries
- Rename "merge init container" to "config generation init container"
- Provider entries added directly to config.yaml providers section

The new approach is simpler:
1. Init containers install packages from container images
2. Config generation adds provider entries with module: field
3. LlamaStack imports via importlib.import_module()

No pip install from external indexes - fully self-contained.

Assisted-By: 🤖 Claude Code
Signed-off-by: Roland Huß <rhuss@redhat.com>
- Add Terminology section for Phase usage disambiguation
- Add kubebuilder validation patterns for providerId and image fields
- Add NFR-006 for configurable volume size limit (default 2Gi)
- Add volumeSizeLimit field to ExternalProvidersSpec CRD
- Clarify FR-020 timing (config generation init container)
- Clarify SC-004 success criteria for provider functionality
- Update lls-preflight-spec open questions to align with main spec

Assisted-by: 🤖 Claude Code
Signed-off-by: Roland Huß <rhuss@redhat.com>
Implements the core functionality for external provider injection at
deployment time, enabling custom LlamaStack providers via container images.

CRD Extensions:
- ExternalProvidersSpec organized by API type (inference, safety, etc.)
- ExternalProviderRef with providerID, image, imagePullPolicy, config
- ExternalProviderStatus for phase-based tracking
- ExternalProviderPhase enum (Pending, Installing, Ready, Failed)

Provider Package (pkg/provider/):
- ProviderMetadata struct matching lls-provider-spec.yaml schema
- LoadProviderMetadata and ParseProviderMetadata functions
- ValidateMetadata with comprehensive validation rules
- API name normalization between CRD and config.yaml formats

Deploy Package (pkg/deploy/):
- GenerateInitContainers for provider installation
- GenerateExtractConfigInitContainer for base config extraction
- Volume helpers for shared emptyDir configuration
- MountExternalProvidersVolume and UpdatePythonPath helpers
- RunYamlConfig struct and MergeExternalProviders function
- GenerateConfig orchestrating the full merge process

Controller Integration:
- configureExternalProviders in resource_helper.go
- Status tracking functions in status.go
- PYTHONPATH and config path updates for main container

Binary (cmd/generate-config/):
- CLI tool for config generation init container
- Reads provider metadata and merges into base config.yaml

Dockerfile:
- Builds and includes generate-config binary

Assisted-By: 🤖 Claude Code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant