Refactor database stats to return raw byte counts and CLI command grouping and support OpenClaw Plugins (#57)

<!-- AI-PR-DESCRIPTION-START -->
## PR Auto Describe
## Summary
This PR adds official support for the OpenClaw agent framework, renames
the deprecated `omni learn --status` flag to `--discover` across the
codebase, improves command savings tracking, and updates release tooling
to sync plugin versions.

---

## Key Changes
- Official OpenClaw integration plugin suite
- Breaking flag rename: `--status` → `--discover` for `omni learn`
- Enhanced stats tracking with command grouping
- Updated release scripts and documentation

---

## Detailed Breakdown
- **OpenClaw Integration**: Added `integrations/openclaw/` with a full
plugin setup: README guide, Node.js tool handlers for `omni_shell`
(wraps `omni exec`) and `omni_rewind` (retrieves full archived logs),
and a plugin manifest with configurable options. Updated
`scripts/bump_version.sh` and `scripts/omni-release.sh` to sync the
plugin's version with the core OMNI binary.
- **Flag Rename**: Replaced all instances of `omni learn --status` with
`--discover` across CLI code (`src/cli/learn.rs`), all documentation
examples, checklists, and help text.
- **Stats & Tracking Improvements**: 
- Updated SQLite store methods to track raw input/output bytes instead
of precomputed reduction percentages for accurate command grouping
- Added command grouping logic in `src/cli/stats.rs` to aggregate
similar commands (e.g., `npm install` variants)
- Fixed `src/hooks/post_tool.rs` to use cleaned command strings when
recording distillation data
- Updated test assertions in `tests/savings_assertions.rs` to match the
new stats API schema
- **Documentation Updates**: Added an OpenClaw integration link to the
root README's user resources, updated the agent flowchart to include
OpenClaw, and revised all `omni learn` examples in `docs/HOW_TO_USE.md`.

---

## Notes (Optional)
The OpenClaw integration reduces terminal output token usage by up to
90% by filtering out noise, while preserving full access to raw logs via
the `omni_rewind` tool when needed.

---

## Breaking Changes (If any)
- The `omni learn --status` flag has been fully removed and replaced
with `--discover`; existing scripts or automation using the old flag
will fail until updated.

_Last updated: 2026-04-12 14:21:53_
<!-- AI-PR-DESCRIPTION-END -->

Author: fajarhide

README.md

@@ -66,7 +66,7 @@ AI agents like Claude are only as smart as the context you feed them. When you f
 
 ```mermaid
 flowchart TB
-    Agent["Claude Code / MCP Agent"]
+    Agent["Claude Code / OpenClaw / MCP Agent"]
     
     subgraph Hooks["Native Hook Layer (Transparent)"]
         Pre["Pre-Hook\n(Rewriter)"]
@@ -151,6 +151,7 @@ You can easily create your own rules using simple TOML files.
 
 **For Users:**
 - [The Ultimate Guide (HOW_TO_USE.md)](docs/HOW_TO_USE.md) — Everything you need: Installation, `omni learn`, Custom TOML Filters, and CLI Commands.
+- [OpenClaw Integration](integrations/openclaw/README.md) — How to use OMNI with the OpenClaw agent framework.
 
 **For Developers & System Integrators:**
 - [Development Guide](docs/DEVELOPMENT.md) — How to build and contribute to the OMNI codebase.

docs/HOW_TO_USE.md

@@ -71,7 +71,7 @@ flowchart TD
     Work --> Learn
 
     subgraph Learn [When Output Isn't Filtered Well]
-        LearnStatus["omni learn --status"] --> DryRun["omni learn --dry-run"]
+        LearnDiscover["omni learn --discover"] --> DryRun["omni learn --dry-run"]
         DryRun --> Apply["omni learn --apply"]
     end
 
@@ -396,7 +396,7 @@ Whenever OMNI runs as a hook (e.g., inside Claude Code), it silently monitors fo
 
 You can process this queue by discovering patterns first:
 ```bash
-omni learn --status
+omni learn --discover
 ```
 
 Then preview or apply:
@@ -409,7 +409,7 @@ omni learn --dry-run
 If you have a log file or a command output that is very noisy, you can pipe it directly into OMNI to generate a filter:
 
 ```bash
-cat build.log | omni learn --status
+cat build.log | omni learn --discover
 ```
 
 ### 3. Applying Learned Filters
@@ -710,7 +710,7 @@ expected = "2024-01-15 10:30:03 ERROR Connection timeout to redis-primary"
 omni learn --verify
 
 # Discovery: search for patterns in a log file
-omni learn --status < output.log
+omni learn --discover < output.log
 
 # Preview: show generated TOML
 omni learn --dry-run < output.log
@@ -953,7 +953,7 @@ omni session --transcript # View transcript of recent session
 Auto-generate TOML filters from passthrough output.
 
 ```bash
-omni learn --status     # Discovery: Search for new noise patterns
+omni learn --discovery     # Discovery: Search for new noise patterns
 omni learn --dry-run    # Preview: Show suggested TOML
 omni learn --apply      # Action: Commit to learned.toml
 omni learn --verify     # Test: Run inline tests on all filters
@@ -1116,7 +1116,7 @@ omni doctor --fix  # Self-repair
 
 ### Weekly / Ongoing
 - [ ] Run `omni stats` to review savings
-- [ ] Run `omni learn --status` to check for new noise patterns to filter
+- [ ] Run `omni learn --discovery` to check for new noise patterns to filter
 - [ ] Run `omni learn --dry-run` if there are patterns
 - [ ] Run `omni learn --apply` if the dry-run looks good
 - [ ] Run `omni learn --verify` after applying any filters

integrations/openclaw/README.md

@@ -0,0 +1,59 @@
+# OMNI Integration for OpenClaw
+
+This plugin integrates **OMNI** into the **OpenClaw** agent framework, allowing OpenClaw agents to intelligently filter terminal output, reducing token consumption by up to 90% while improving context quality.
+
+## Prerequisites
+
+- **OMNI** must be installed and available in your PATH.
+- **OpenClaw** (Gateway) must be installed.
+
+## Installation
+
+1. Clone or copy the OMNI repository.
+2. Navigate to the OMNI directory.
+3. Install the plugin into OpenClaw:
+
+```bash
+openclaw plugins install ./integrations/openclaw
+```
+
+## Configuration (Optional)
+
+**OMNI for OpenClaw is designed to be "Zero Config".** If `omni` is already in your `PATH`, it will work immediately after installation without any additional settings.
+
+If you have a custom setup, you can modify your OpenClaw settings (`~/.openclaw/config.yaml`):
+
+```yaml
+plugins:
+  omni-signal-engine:
+    omniPath: "/usr/local/bin/omni"  # Optional: path to omni binary
+    forceDistill: false             # Optional: experimental override
+```
+
+## Usage
+
+Once installed, your OpenClaw agent will have access to two new tools:
+
+### `omni_shell`
+Use this exactly like the standard `shell` or `bash` tool. 
+- **Input**: `{ "command": "npm install" }`
+- **Behavior**: Runs the command via `omni exec`, filtering out noise and keeping only the signal (errors, summaries).
+
+### `omni_rewind`
+If OMNI omits a large block of text and provides a hash (e.g., `[OMNI: 847 lines omitted — hash: a3f8c2d1]`), the agent can call `omni_rewind` with that hash to see the full output.
+- **Input**: `{ "hash": "a3f8c2d1" }`
+
+## Monitoring Savings
+
+You can track how many tokens and how much money the OpenClaw plugin is saving you by running the following command in your terminal:
+
+```bash
+omni stats --today
+```
+
+This will show a detailed breakdown of all commands processed for your OpenClaw session, including the signal reduction percentage.
+
+## Benefits
+- **Cheaper Tasks**: Massive savings on API bills for long-running autonomous tasks.
+- **Higher Accuracy**: Agents focus on the real errors instead of being distracted by 10,000 lines of build logs.
+- **Zero Information Loss**: The agent can always "rewind" to see the full raw logs if needed.

integrations/openclaw/index.js

@@ -0,0 +1,88 @@
+const { exec } = require('child_process');
+const util = require('util');
+const execPromise = util.promisify(exec);
+
+/**
+ * OMNI Plugin for OpenClaw
+ * 
+ * Provides a highly-efficient shell tool that uses OMNI's
+ * distillation engine to reduce token usage.
+ */
+module.exports = function(sdk) {
+  if (!sdk) return;
+
+  const config = (typeof sdk.getConfig === 'function') ? sdk.getConfig() : {};
+  const omniPath = config.omniPath || 'omni';
+
+  sdk.registerTool({
+    id: "omni_shell",
+    description: "Run a shell command with intelligent OMNI filtering. Use this for noisy commands (git, npm, cargo, docker) to save 80-90% of token costs.",
+    schema: {
+      type: "object",
+      properties: {
+        command: {
+          type: "string",
+          description: "The shell command to execute"
+        }
+      },
+      required: ["command"]
+    },
+    handler: async ({ command }) => {
+      try {
+        // We wrap the command in `omni exec`
+        // The -- ensures that omni doesn't parse command flags as its own
+        const fullCommand = `${omniPath} exec -- ${command}`;
+        
+        const { stdout, stderr } = await execPromise(fullCommand);
+        
+        let result = stdout || "";
+        if (stderr && stderr.trim()) {
+          result += `\n[stderr]\n${stderr}`;
+        }
+
+        return {
+          content: result || "(No output from OMNI)",
+          role: "tool"
+        };
+      } catch (error) {
+        sdk.log(`OMNI Error: ${error.message}`, "error");
+        return {
+          content: `Error running OMNI: ${error.message}\n${error.stderr || ""}`,
+          isError: true
+        };
+      }
+    }
+  });
+
+  // Optional: Provide a direct rewind tool
+  sdk.registerTool({
+    id: "omni_rewind",
+    description: "Retrieve full archived output from OMNI if the distilled summary was insufficient.",
+    schema: {
+      type: "object",
+      properties: {
+        hash: {
+          type: "string",
+          description: "The 8-character hash provided in the OMNI summary"
+        }
+      },
+      required: ["hash"]
+    },
+    handler: async ({ hash }) => {
+      try {
+        const { stdout } = await execPromise(`${omniPath} rewind ${hash}`);
+        return {
+          content: stdout || "No archive found for this hash.",
+          role: "tool"
+        };
+      } catch (error) {
+        return {
+          content: `Failed to retrieve OMNI archive: ${error.message}`,
+          isError: true
+        };
+      }
+    }
+  });
+
+  sdk.log("OMNI Signal Engine plugin loaded successfully.");
+};

integrations/openclaw/openclaw.plugin.json

@@ -0,0 +1,22 @@
+{
+  "id": "omni-signal-engine",
+  "name": "OMNI Signal Engine",
+  "version": "0.5.6-rc1",
+  "description": "Intelligent context filtering for OpenClaw using OMNI. Saves tokens by distilling shell output.",
+  "author": "OMNI Team",
+  "configSchema": {
+    "type": "object",
+    "properties": {
+      "omniPath": {
+        "type": "string",
+        "description": "Path to the omni binary (defaults to 'omni' in PATH)",
+        "default": "omni"
+      },
+      "forceDistill": {
+        "type": "boolean",
+        "description": "If true, overrides the default shell tool (experimental)",
+        "default": false
+      }
+    }
+  }
+}

scripts/bump_version.sh

@@ -31,11 +31,15 @@ rm -f Cargo.toml.bak
 # 3. Update Cargo.lock
 cargo check --quiet 2>/dev/null || true
 
-# 4. Verify build
+# 4. Update openclaw plugin version
+sed -i.bak "s/^version = \".*\"/version = \"$NEW\"/" integrations/openclaw/openclaw.plugin.json
+rm -f integrations/openclaw/openclaw.plugin.json.bak
+
+# 5. Verify build
 echo "Verifying build..."
 cargo build --quiet
 
-# 5. Verify version output
+# 6. Verify version output
 ACTUAL=$(./target/debug/omni version 2>&1)
 if echo "$ACTUAL" | grep -q "$NEW"; then
     echo "✓ Version output: $ACTUAL"
@@ -45,7 +49,7 @@ else
 fi
 
 # 6. Stage and commit
-git add Cargo.toml Cargo.lock
+git add Cargo.toml Cargo.lock integrations/openclaw/openclaw.plugin.json
 git commit -m "chore: bump version to $NEW"
 
 echo ""

scripts/omni-release.sh

@@ -34,13 +34,20 @@ if [ "$CARGO_VERSION" != "$VERSION" ]; then
 fi
 ok "Cargo.toml version: $CARGO_VERSION"
 
-# 2. Git status check
+# 2. Verify version in openclaw.plugin.json
+OPENCLAW_VERSION=$(grep '^version' integrations/openclaw/openclaw.plugin.json | head -1 | sed 's/version = "\(.*\)"/\1/')
+if [ "$OPENCLAW_VERSION" != "$VERSION" ]; then
+    fail "openclaw.plugin.json version ($OPENCLAW_VERSION) ≠ release version ($VERSION)\n  Fix: edit openclaw.plugin.json version field, or run: ./scripts/bump_version.sh $VERSION"
+fi
+ok "openclaw.plugin.json version: $OPENCLAW_VERSION"
+
+# 3. Git status check
 if ! git diff --quiet HEAD; then
     fail "Working directory is not clean. Commit or stash changes first."
 fi
 ok "Git working directory: clean"
 
-# 3. Branch check
+# 4. Branch check
 BRANCH=$(git branch --show-current)
 if [ "$BRANCH" != "main" ] && [ "$BRANCH" != "homebrew_fix" ]; then
     warn "Not on main branch (on: $BRANCH). Continue? [y/N]"
@@ -49,7 +56,7 @@ if [ "$BRANCH" != "main" ] && [ "$BRANCH" != "homebrew_fix" ]; then
 fi
 ok "Branch: $BRANCH"
 
-# 4. Tag check
+# 5. Tag check
 if git tag --list | grep -q "^${TAG}$"; then
     fail "Tag ${TAG} already exists"
 fi
@@ -60,21 +67,21 @@ ok "Tag ${TAG}: not yet created"
 echo ""
 info "Build validation"
 
-# 5. cargo fmt check
+# 6. cargo fmt check
 cargo fmt --check || fail "cargo fmt check failed. Run: cargo fmt"
 ok "cargo fmt: clean"
 
-# 6. cargo clippy
+# 7. cargo clippy
 echo "   Running clippy..."
 cargo clippy --all-targets -- -D warnings > /tmp/omni-clippy 2>&1 || { cat /tmp/omni-clippy && fail "clippy warnings found"; }
 ok "cargo clippy: no warnings"
 
-# 7. cargo test
+# 8. cargo test
 echo "   Running tests..."
 cargo test --all > /tmp/omni-test 2>&1 || { tail -n 20 /tmp/omni-test && fail "tests failed"; }
 ok "cargo test: all pass"
 
-# 8. Release build
+# 9. Release build
 echo "   Building release..."
 cargo build --release > /tmp/omni-build 2>&1 || { tail -n 20 /tmp/omni-build && fail "release build failed"; }
 BINARY_SIZE=$(du -k target/release/omni | cut -f1)
@@ -83,7 +90,7 @@ if [ "$BINARY_SIZE" -gt 5120 ]; then
     warn "Binary size ${BINARY_SIZE}KB exceeds 5MB target"
 fi
 
-# 9. Smoke test
+# 10. Smoke test
 if [ -x tests/smoke_test.sh ]; then
     echo "   Running smoke tests..."
     bash tests/smoke_test.sh ./target/release/omni > /tmp/omni-smoke 2>&1 || { cat /tmp/omni-smoke && fail "smoke test failed"; }

src/cli/learn.rs

@@ -18,7 +18,7 @@ fn print_help() {
     println!("\n{}", "FLAGS:".bold().bright_white());
     println!(
         "  {: <12} Discover and view candidate patterns",
-        "--status".cyan()
+        "--discover".cyan()
     );
     println!(
         "  {: <12} Automatically append new filters to config",
@@ -40,7 +40,7 @@ fn print_help() {
 
     println!("\n{}", "EXAMPLES:".bold().bright_white());
     println!(
-        "  omni learn --status   {}",
+        "  omni learn --discover {}",
         "# Search for new noise patterns".bright_black()
     );
     println!(
@@ -75,10 +75,10 @@ pub fn run_learn(args: &[String]) -> Result<()> {
     let dry_run = args.iter().any(|a| a == "--dry-run");
     let from_queue = args.iter().any(|a| a == "--from-queue");
     let verify = args.iter().any(|a| a == "--verify");
-    let is_status = args.iter().any(|a| a == "--status");
+    let is_discover = args.iter().any(|a| a == "--discover");
 
     // If no flags, show help
-    if !apply && !dry_run && !from_queue && !verify && !is_status {
+    if !apply && !dry_run && !from_queue && !verify && !is_discover {
         print_help();
         return Ok(());
     }
@@ -117,7 +117,7 @@ pub fn run_learn(args: &[String]) -> Result<()> {
 
     // In terminal mode, if an action is used without --from-queue, default to queue
     let mut use_queue = from_queue;
-    let is_action = is_status || dry_run || apply;
+    let is_action = is_discover || dry_run || apply;
     if is_action && !from_queue && io::stdin().is_terminal() {
         use_queue = true;
     }