Skip to content

feat: support model inclusion and exclusion in use case descriptions #145

@anfredette

Description

@anfredette

Summary

Allow users to specify which models to include or exclude when describing their use case in natural language (e.g., "I want to use Llama 3.1 or Qwen3" or "don't recommend Phi models").

Current Behavior

  • Users can specify GPU preferences via preferred_gpu_types in the intent schema
  • There is no equivalent mechanism for filtering models by name or family
  • The recommendation engine considers all models in the catalog that match the use case

Desired Behavior

Users should be able to express model preferences in natural language like:

  • "I want to use Llama 3.1 70B or Qwen3 8B"
  • "Don't recommend any Phi models"
  • "Only consider Llama models"
  • "Use Mistral or Llama but not Qwen"

Proposed Implementation

1. Schema changes (src/neuralnav/shared/schemas/intent.py)

Add two new fields to DeploymentIntent:

  • model_include: list[str] — Models or model families to include (empty = all)
  • model_exclude: list[str] — Models or model families to exclude

2. LLM prompt changes (src/neuralnav/llm/prompts.py)

Update the intent extraction prompt and schema to extract model preferences from natural language input.

3. Recommendation engine filtering

Apply model include/exclude filtering during the recommendation pipeline. Options include:

  • Adding filter conditions to the database query in find_configurations_meeting_slo()
  • Post-query filtering of benchmark results before scoring
  • Filtering during scoring in config_finder.py

The best approach depends on where it's most natural to apply — the implementation should determine this.

Related Issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions