Commit 0531a53
committed
fix: Prevent incorrect priority inference from use case and hardware
Problem:
- "Customer service chatbot for 30 users" incorrectly extracted latency_priority: "low"
- "chatbot with H100/H200 gpus" incorrectly extracted cost_priority: "low"
- LLM was inferring priorities from use case type and hardware choices instead
of explicit user statements
Root causes:
1. Experience class guidance contained "low latency needed" text, confusing LLM
into thinking latency_priority should be "low" (deprioritized) instead of
understanding it as a functional requirement
2. LLM inferred "H100 = expensive GPU = user doesn't care about cost" and set
cost_priority: "low"
3. Weak prompt guidance allowed use case and hardware to influence priorities
Changes:
1. backend/src/llm/prompts.py:
- Remove latency-related language from Experience class guidance to prevent
confusion between functional requirements and priority weights
- Strengthen cost_priority extraction with "EXPLICITLY" and "ONLY" keywords
- Add "DO NOT infer from GPU choice" warning to prevent hardware → cost inference
- Strengthen latency_priority extraction to require explicit user statements
- Add "FOLLOW STRICTLY" section with clear rules and 4 concrete examples
- Default to "medium" for all priorities unless user explicitly states otherwise
2. ui/app.py:
- Fix rendering bug where empty priorities_item string caused "</div>" to
display as visible text
- Use HTML comment placeholder instead of empty string
Testing:
- "Customer service chatbot for 30 users" → all priorities "medium" ✓
- "chatbot with 300 users, low latency important, H100 gpus" →
latency_priority: "high", cost_priority: "medium" ✓
Assisted-by: Claude <noreply@anthropic.com>
Signed-off-by: Andre Fredette <afredette@redhat.com>1 parent d327199 commit 0531a53
2 files changed
+19
-13
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
30 | | - | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| |||
87 | 87 | | |
88 | 88 | | |
89 | 89 | | |
90 | | - | |
91 | | - | |
| 90 | + | |
| 91 | + | |
92 | 92 | | |
93 | | - | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
94 | 103 | | |
95 | 104 | | |
96 | 105 | | |
97 | 106 | | |
98 | 107 | | |
99 | | - | |
100 | | - | |
101 | | - | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4877 | 4877 | | |
4878 | 4878 | | |
4879 | 4879 | | |
4880 | | - | |
| 4880 | + | |
4881 | 4881 | | |
4882 | 4882 | | |
4883 | 4883 | | |
| |||
4934 | 4934 | | |
4935 | 4935 | | |
4936 | 4936 | | |
4937 | | - | |
| 4937 | + | |
4938 | 4938 | | |
4939 | 4939 | | |
4940 | 4940 | | |
| |||
0 commit comments