Skip to content

Commit a2fd2fd

Browse files
author
Neo
committed
Trinity Part 3: tune escalation criteria to on-device capability
The on-device Apple model is strong on conversation / everyday Q&A / short explanations / tool-calls, but unreliable on multi-step math, comparisons, precise facts, and logic — and those questions are often SHORT, so the length + deep-marker triggers missed them (a short word-problem routed local and the small model muddled the arithmetic). Added correctnessSensitiveMarkers ('calculate', 'how many', 'which is bigger', 'what year', 'when did', …) so correctness-sensitive prompts escalate to the cloud when reachable (best-effort on-device when not — never a fabricated answer). Conversation/opinion/creative/light how-to stays local. Honest heuristic: Apple's on-device API exposes no reliable confidence signal. Build green.
1 parent 872816a commit a2fd2fd

1 file changed

Lines changed: 23 additions & 1 deletion

File tree

Core/Trinity/InferenceRouter.swift

Lines changed: 23 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -892,7 +892,11 @@ struct FoundationModelsReasoningProvider: OfflineReasoningProvider {
892892
/// for large context the small on-device model handles poorly), OR
893893
/// • the prompt carries a deep/multi-step/expert signal
894894
/// (`deepReasoningMarkers`, e.g. "step by step", "prove", "in depth",
895-
/// "write an essay", "research").
895+
/// "write an essay", "research"), OR
896+
/// • the prompt is CORRECTNESS-SENSITIVE (`correctnessSensitiveMarkers`,
897+
/// e.g. "calculate", "how many", "which is bigger", "what year", "when
898+
/// did") — often SHORT, but where the small on-device model is unreliable
899+
/// and a wrong answer is costly, so it escalates rather than guessing.
896900
/// 3. Otherwise → `.onDevice` if available (the default), else `.escalateToCloud`
897901
/// if the cloud is reachable, else `.honestlyUnavailable`.
898902
///
@@ -913,6 +917,23 @@ struct ReasoningRouter {
913917
"write an essay", "write a report", "long-form", "research",
914918
]
915919

920+
/// Correctness-sensitive markers: questions where a WRONG answer is costly and
921+
/// the small on-device model is unreliable — multi-step math, comparisons,
922+
/// precise facts, logic. These are often SHORT (so length alone misses them),
923+
/// which is why they get their own trigger. Escalated to the cloud when reachable.
924+
/// Kept phrase-based and narrow so ordinary conversation / opinion / creative /
925+
/// light how-to stays local. Heuristic by necessity: Apple's on-device API
926+
/// exposes no reliable confidence signal, so the router errs toward the cloud for
927+
/// correctness-sensitive content WHEN a cloud is available, and best-efforts
928+
/// locally when it isn't (better than nothing, and never a fabricated answer).
929+
static let correctnessSensitiveMarkers: [String] = [
930+
"calculate", "how many", "how much", "what percent", "percent of",
931+
"solve for", "solve this", "equation", "which is bigger", "which is larger",
932+
"which is faster", "which is more", "what year", "when did", "who invented",
933+
"how far is", "how long does", "how long until", "reason through",
934+
"logic puzzle", "riddle", "multi-step",
935+
]
936+
916937
func route(prompt: String,
917938
onDeviceAvailable: Bool,
918939
cloudReachable: Bool,
@@ -927,6 +948,7 @@ struct ReasoningRouter {
927948
let exceedsLocal = forceCloud
928949
|| prompt.count > localCharBudget
929950
|| Self.deepReasoningMarkers.contains { lower.contains($0) }
951+
|| Self.correctnessSensitiveMarkers.contains { lower.contains($0) }
930952
if exceedsLocal {
931953
if cloudReachable { return .escalateToCloud }
932954
if onDeviceAvailable { return .onDevice } // best-effort beats nothing

0 commit comments

Comments
 (0)