Summary
Allow users to specify which models to include or exclude when describing their use case in natural language (e.g., "I want to use Llama 3.1 or Qwen3" or "don't recommend Phi models").
Current Behavior
- Users can specify GPU preferences via
preferred_gpu_types in the intent schema
- There is no equivalent mechanism for filtering models by name or family
- The recommendation engine considers all models in the catalog that match the use case
Desired Behavior
Users should be able to express model preferences in natural language like:
- "I want to use Llama 3.1 70B or Qwen3 8B"
- "Don't recommend any Phi models"
- "Only consider Llama models"
- "Use Mistral or Llama but not Qwen"
Proposed Implementation
1. Schema changes (src/neuralnav/shared/schemas/intent.py)
Add two new fields to DeploymentIntent:
model_include: list[str] — Models or model families to include (empty = all)
model_exclude: list[str] — Models or model families to exclude
2. LLM prompt changes (src/neuralnav/llm/prompts.py)
Update the intent extraction prompt and schema to extract model preferences from natural language input.
3. Recommendation engine filtering
Apply model include/exclude filtering during the recommendation pipeline. Options include:
- Adding filter conditions to the database query in
find_configurations_meeting_slo()
- Post-query filtering of benchmark results before scoring
- Filtering during scoring in
config_finder.py
The best approach depends on where it's most natural to apply — the implementation should determine this.
Related Issues
Summary
Allow users to specify which models to include or exclude when describing their use case in natural language (e.g., "I want to use Llama 3.1 or Qwen3" or "don't recommend Phi models").
Current Behavior
preferred_gpu_typesin the intent schemaDesired Behavior
Users should be able to express model preferences in natural language like:
Proposed Implementation
1. Schema changes (
src/neuralnav/shared/schemas/intent.py)Add two new fields to
DeploymentIntent:model_include: list[str]— Models or model families to include (empty = all)model_exclude: list[str]— Models or model families to exclude2. LLM prompt changes (
src/neuralnav/llm/prompts.py)Update the intent extraction prompt and schema to extract model preferences from natural language input.
3. Recommendation engine filtering
Apply model include/exclude filtering during the recommendation pipeline. Options include:
find_configurations_meeting_slo()config_finder.pyThe best approach depends on where it's most natural to apply — the implementation should determine this.
Related Issues