Implement command-aware content classification heuristics (#51)

## PR Auto Describe
## Summary
This PR delivers a major overhaul of omni's content processing pipeline,
including command-aware content classification, historical data repair
via `omni doctor`, pipeline refactoring, and CLI tooling improvements to
boost detection accuracy and user experience.

---

## Key Changes
- **Command-Aware Content Classification**: Rewrote the core `classify`
function to use command context for far more accurate content type
detection, with built-in heuristics for git, system, build, web, and
cloud commands.
- **Historical Stats Repair**: Added `omni doctor --fix` to reclassify
outdated, mislabeled historical distillation records in the local SQLite
store.
- **Pipeline Refactor**: Simplified hook and pipeline logic, replaced
the legacy `compose` function with dedicated distillation and auto-learn
evaluation code, added output truncation safety.
- **CLI & Tooling Updates**: Refined `omni stats` with better filtering
and clearer messaging, updated `omni learn` to include command context
in generated filters, reduced update check cache TTL to 4 hours.

---

## Detailed Breakdown
### Core Classification System
Rewrote `pipeline/classifier.rs` to add an optional `command_name`
parameter to `classify()`, with pre-stage heuristics that extract the
base command (e.g. `git` from `git status`) and route outputs to correct
content types: git subcommands, system utilities, build tools, web/JS
tools, and cloud tools. Updated all existing tests to use the new API,
added new test cases for command-specific classification, and updated
all benchmarks/tests to pass `None` as context where unavailable.

### Pipeline & Hook Logic
Refactored `hooks/pipe.rs` and `hooks/post_tool.rs` to pass command
context to `classify()`, replaced legacy `compose` calls with direct
distiller usage, added auto-learn evaluation via `evaluate_learning()`,
and added output truncation safety checks. Split `composer.rs`'s
`compose` function into a dedicated `evaluate_learning()` function for
auto-learn triggers, removed the unused `colored` import, and deleted
the legacy `compose` function entirely. Added properly formatted rewind
store notifications in both hook files.

### Data Store & Doctor Command
Added `reclassify_historical_data()` and `has_upgradable_history()` to
`sqlite.rs` to update old misclassified distillation records using their
stored command context. Updated `cli/doctor.rs` to add an Intelligence
Consistency check section: prompts users to run `--fix` to upgrade
historical stats, or runs repairs automatically when the flag is
present. Updated `stats.rs` to show a prompt to run `omni doctor --fix`
when upgradable history exists, added filter flag support
(--today/--week/--month/--all-commands) that defaults to detail mode,
improved messaging for hidden zero-savings commands, and cleaned up
default stats labeling.

### CLI Learn & MCP Server
Updated `cli/learn.rs` to pass command context to `generate_toml()` and
`apply_to_config()`, so auto-generated filters include the triggering
command as a `match_command` rule and better descriptions. Updated
`mcp/server.rs` to use the new `classify()` API and pass None context,
updated filter generation calls to match the new signature.

### Minor Updates
Reduced the update check cache TTL in `guard/update.rs` from 24 hours to
4 hours. Updated all test files (`savings_assertions.rs`,
`security_tests.rs`) to use the new `classify()` API and replace legacy
`compose` calls with direct distiller usage. Updated all benchmark calls
in `benches/pipeline.rs` to pass the new command context parameter.

---

## Notes
- The command-aware classification drastically improves accuracy for
tool-specific output (e.g. git diffs, cargo build logs, kubectl outputs)
by using the original executed command as context.
- The historical reclassification fixes long-standing issues with
mislabeled old stats records.

---

## Breaking Changes
1. **`classifier::classify` API Break**: The function now requires an
optional `command_name` parameter; all existing calls must be updated to
pass `None` if no command context is available (e.g.
`classifier::classify(input, None)`).
2. **Removed `composer::compose` Function**: Direct calls to this legacy
function will fail; use the distiller directly alongside
`evaluate_learning()` for auto-learn logic.
3. **Updated `generate_toml` and `apply_to_config` Signatures**: These
functions now accept an optional `command` parameter; callers must add
this argument (pass `None` if unused).

Author: fajarhide

benches/pipeline.rs

@@ -25,7 +25,7 @@ fn bench_classify(c: &mut Criterion) {
 
     for (name, input) in &fixtures {
         c.bench_with_input(BenchmarkId::new("classify", name), input, |b, i| {
-            b.iter(|| classifier::classify(i))
+            b.iter(|| classifier::classify(i, None))
         });
     }
 }
@@ -35,7 +35,7 @@ fn bench_full_pipeline(c: &mut Criterion) {
 
     c.bench_function("full_pipeline_cargo_build", |b| {
         b.iter(|| {
-            let ctype = classifier::classify(input);
+            let ctype = classifier::classify(input, None);
             let segments = scorer::score_segments(input, &ctype, None);
             let distiller = distillers::get_distiller(&ctype);
             distiller.distill(&segments, input, None)
@@ -49,7 +49,7 @@ fn bench_hook_roundtrip(c: &mut Criterion) {
 
     c.bench_function("hook_roundtrip_50kb", |b| {
         b.iter(|| {
-            let ctype = classifier::classify(&large_input);
+            let ctype = classifier::classify(&large_input, None);
             let segments = scorer::score_segments(&large_input, &ctype, None);
             let distiller = distillers::get_distiller(&ctype);
             distiller.distill(&segments, &large_input, None)

src/cli/doctor.rs

@@ -430,6 +430,47 @@ pub fn run(args: &[String]) -> anyhow::Result<()> {
         println!("   {:<15} none", "Project:".bright_black());
     }
 
+    // 10. Intelligence Consistency
+    println!("\n {}", "Intelligence:".bold().bright_white());
+    if let Ok(store) = Store::open() {
+        if fix_mode {
+            match store.reclassify_historical_data() {
+                Ok(count) => {
+                    if count > 0 {
+                        println!(
+                            "   {:<15} {} records upgraded to new categories {}",
+                            "Upgrade:".bright_black(),
+                            count.to_string().yellow().bold(),
+                            "[FIXED]".green().bold()
+                        );
+                    } else {
+                        println!(
+                            "   {:<15} historical statistics are up to date {}",
+                            "Status:".bright_black(),
+                            "[OK]".green().bold()
+                        );
+                    }
+                }
+                Err(e) => {
+                    println!(
+                        "   {:<15} upgrade failed: {} {}",
+                        "Upgrade:".bright_black(),
+                        e,
+                        "[ERROR]".red().bold()
+                    );
+                }
+            }
+        } else {
+            // Diagnostic mode: check how many could be upgraded
+            // For now, simpler to just say 'Run with --fix to upgrade'
+            println!(
+                "   {:<15} Run with {} to upgrade historical stats",
+                "Status:".bright_black(),
+                "--fix".cyan()
+            );
+        }
+    }
+
     if let Some(latest) = crate::guard::update::check() {
         crate::guard::update::print_notification(&latest);
     }

src/cli/learn.rs

@@ -236,8 +236,11 @@ pub fn run_learn(args: &[String]) -> Result<()> {
 
     let filter_name = format!("learned_{}", Utc::now().timestamp());
 
+    let command_hint = executions.first().map(|e| e.command.as_str());
+
     if dry_run {
-        let generated = crate::session::learn::generate_toml(&candidates, &filter_name);
+        let generated =
+            crate::session::learn::generate_toml(&candidates, &filter_name, command_hint);
         println!(
             "\n{}",
             "─────────────────────────────────────────"
@@ -264,7 +267,7 @@ pub fn run_learn(args: &[String]) -> Result<()> {
     } else if apply {
         let path = crate::paths::learned_filters_path();
         let _ = crate::paths::ensure_omni_home();
-        let added = apply_to_config(&candidates, &filter_name, &path)?;
+        let added = apply_to_config(&candidates, &filter_name, &path, command_hint)?;
         if added > 0 {
             println!(
                 "\n{}",

src/cli/stats.rs

@@ -137,13 +137,21 @@ pub fn run(args: &[String], store: &Store) -> Result<()> {
         return Ok(());
     }
 
-    // Mode detection
-    let mode = if args.iter().any(|a| a == "--detail") {
+    let detail_flag = args.iter().any(|a| a == "--detail");
+    let type_flag = args.iter().any(|a| a == "--by-type");
+    let json_flag = args.iter().any(|a| a == "--json");
+    let filter_flag = args
+        .iter()
+        .any(|a| a == "--today" || a == "--week" || a == "--month" || a == "--all-commands");
+
+    let mode = if detail_flag {
         "detail"
-    } else if args.iter().any(|a| a == "--by-type") {
+    } else if type_flag {
         "by-type"
-    } else if args.iter().any(|a| a == "--json") {
+    } else if json_flag {
         "json"
+    } else if filter_flag {
+        "detail" // Implicit detail mode for scoped queries
     } else {
         "default"
     };
@@ -227,7 +235,7 @@ fn run_default(store: &Store) -> Result<()> {
 
     if !top_types.is_empty() {
         println!("\n  {}", "Top Savings by Type:".bold().bright_white());
-        for (content_type, count, pct, _) in &top_types {
+        for (content_type, count, pct, _commands) in &top_types {
             let bar = format_bar_with_empty(*pct);
             let bar_colored = if *pct > 80.0 {
                 bar.bright_green()
@@ -237,12 +245,14 @@ fn run_default(store: &Store) -> Result<()> {
                 bar.bright_red()
             };
 
+            let label_display = content_type.clone();
+
             println!(
                 "    {:<13} {}  {:>5.1}%  ({}x)",
-                content_type.bright_cyan(),
+                label_display.bright_cyan(),
                 bar_colored,
                 pct,
-                count,
+                count
             );
         }
     }
@@ -267,6 +277,19 @@ fn run_default(store: &Store) -> Result<()> {
         "  💡 {} for content type mapping",
         "omni stats --by-type".bright_cyan()
     );
+
+    if store.has_upgradable_history() {
+        println!(
+            "  💡 Run {} to upgrade historical stats",
+            "omni doctor --fix".bright_cyan()
+        );
+    }
+
+    // Update Notification (4h cache)
+    if let Some(latest) = crate::guard::update::check() {
+        crate::guard::update::print_notification(&latest);
+    }
+
     println!();
     Ok(())
 }
@@ -368,13 +391,18 @@ fn run_detail(args: &[String], store: &Store) -> Result<()> {
         );
     }
 
-    // By Command — top 10, filter 0% savings
+    // By Command — top 10 (or all if requested), filter 0% savings
     let filters = store.filter_breakdown(since)?;
-    let display_filters: Vec<_> = filters
-        .iter()
-        .filter(|(_, _, pct)| *pct > 0.0)
-        .take(10)
-        .collect();
+    let all_flag = args.iter().any(|a| a == "--all-commands");
+    let display_filters: Vec<_> = if all_flag {
+        filters.iter().collect()
+    } else {
+        filters
+            .iter()
+            .filter(|(_, _, pct)| *pct > 0.0)
+            .take(10)
+            .collect()
+    };
 
     if !display_filters.is_empty() {
         println!("\n {}", "By Command:".bold().bright_white());
@@ -416,16 +444,30 @@ fn run_detail(args: &[String], store: &Store) -> Result<()> {
             );
         }
 
-        if filters.len() > 10 {
-            println!(
-                "\n   {}",
-                format!(
-                    "Run `omni stats --detail --all-commands` for all {} commands.",
-                    filters.len()
-                )
-                .bright_black()
-                .italic()
-            );
+        if !all_flag {
+            let filtered_count = filters.iter().filter(|(_, _, pct)| *pct > 0.0).count();
+            let hidden_zero = filters.len() - filtered_count;
+
+            if filtered_count > 10 {
+                println!(
+                    "\n   {}",
+                    format!(
+                        "Showing top 10 of {} commands with active savings.",
+                        filtered_count
+                    )
+                    .bright_black()
+                    .italic()
+                );
+            }
+
+            if hidden_zero > 0 {
+                println!(
+                     "   {}",
+                     format!("({} noise commands with 0% savings hidden. Use --all-commands to see all).", hidden_zero)
+                         .bright_black()
+                         .italic()
+                 );
+            }
         }
     }
 

src/guard/update.rs

@@ -32,10 +32,10 @@ pub fn get_status() -> Status {
         .unwrap_or_default()
         .as_secs();
 
-    // Try to get latest version from cache or fetch it
+    // Try to get latest version from cache or fetch it (Cache: 4 hours)
     let latest = if let Ok(content) = fs::read_to_string(&cache_path)
         && let Ok(cache) = serde_json::from_str::<UpdateCache>(&content)
-        && now < cache.last_checked + 86400
+        && now < cache.last_checked + 14400
     {
         Some(cache.latest_version)
     } else {

src/hooks/pipe.rs

@@ -179,47 +179,65 @@ fn distill(
         }
     }
 
-    let (output, filter_name, ctype, rewind_hash, kept_count, dropped_count) =
-        if let Some(filter) = matched_toml {
-            let out = filter.apply(&input_text);
-            (out, filter.name.clone(), ContentType::Unknown, None, 0, 0)
-        } else {
-            let c = classifier::classify(&input_text);
-
-            let collapse_result = collapse::collapse(&input_text, &c);
-            let effective_input = collapse_result.collapsed_lines.join("\n");
-
-            let active_session_opt = session.as_ref().and_then(|m| m.lock().ok());
-            let scored_segments =
-                scorer::score_segments(&effective_input, &c, active_session_opt.as_deref());
-            drop(active_session_opt);
-
-            let compose_config = composer::ComposeConfig::default();
-            let decision = composer::decide_rewind(&scored_segments, &c);
-
-            let k_count = scored_segments
-                .iter()
-                .filter(|s| s.final_score() >= compose_config.threshold)
-                .count();
-            let d_count = scored_segments.len() - k_count;
-
-            let store_for_compose = if decision.should_store { store } else { None };
-
-            let (out, r_hash) = composer::compose(
-                scored_segments,
-                if decision.should_store {
-                    Some(input_text.clone())
-                } else {
-                    None
-                }, // Temporary clone for compose drops
-                &compose_config,
-                store_for_compose,
-                &input_text,
-                &c,
-            );
+    let (output, filter_name, ctype, rewind_hash, kept_count, dropped_count) = if let Some(filter) =
+        matched_toml
+    {
+        let out = filter.apply(&input_text);
+        (out, filter.name.clone(), ContentType::Unknown, None, 0, 0)
+    } else {
+        let c = classifier::classify(&input_text, command_name);
 
-            (out, format!("{:?}", c), c, r_hash, k_count, d_count)
-        };
+        let collapse_result = collapse::collapse(&input_text, &c);
+        let effective_input = collapse_result.collapsed_lines.join("\n");
+
+        let active_session_opt = session.as_ref().and_then(|m| m.lock().ok());
+        let scored_segments =
+            scorer::score_segments(&effective_input, &c, active_session_opt.as_deref());
+
+        let distiller = crate::distillers::get_distiller(&c);
+        let mut out =
+            distiller.distill(&scored_segments, &input_text, active_session_opt.as_deref());
+
+        let compose_config = composer::ComposeConfig::default();
+        let decision = composer::decide_rewind(&scored_segments, &c);
+
+        let k_count = scored_segments
+            .iter()
+            .filter(|s| s.final_score() >= compose_config.threshold)
+            .count();
+        let d_count = scored_segments.len() - k_count;
+
+        crate::pipeline::composer::evaluate_learning(
+            &c,
+            &input_text,
+            scored_segments.len(),
+            d_count,
+            command_name.unwrap_or(""),
+        );
+
+        let mut r_hash = None;
+        if decision.should_store
+            && let Some(s) = store
+        {
+            let hash = s.store_rewind(&input_text);
+            out.push_str(&format!(
+                "\n{} {} {} {} lines. The hash {} stores the full output in RewindStore for retrieval.\n",
+                "⏺".cyan(),
+                "OMNI".bold().bright_white(),
+                "distilled".bright_green(),
+                d_count,
+                hash.cyan().bold()
+            ));
+            r_hash = Some(hash);
+        }
+
+        if out.len() > compose_config.max_output_chars {
+            out.truncate(compose_config.max_output_chars);
+            out.push_str("\n[OMNI: output truncated]\n");
+        }
+
+        (out, format!("{:?}", c), c, r_hash, k_count, d_count)
+    };
 
     PipelineResult {
         session_id,