Skip to content

feat: use llm-d well-lit path for generating deployment configurations #144

@anfredette

Description

@anfredette

Summary

Replace the current KServe/vLLM Jinja2 templates with llm-d's well-lit path for generating deployment configurations. This aligns the planner with the llm-d ecosystem and ensures generated configurations follow production-tested patterns.

Context

The current deployment configuration generator (src/neuralnav/configuration/generator.py) uses custom Jinja2 templates (src/neuralnav/configuration/templates/) to produce KServe InferenceService YAML. The llm-d project provides a well-lit path for deployment configuration that should be used instead, ensuring compatibility and best practices.

Scope

  • Integrate the llm-d well-lit path configuration generation
  • Replace or adapt existing Jinja2 templates to use llm-d patterns
  • Ensure generated configs are compatible with llm-d deployments
  • Update the Configuration Service to use the new generation path

Related Issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions