Skip to content

feat(brain): add DarkRadar fifth signal to BrainHub#1

Closed
rudi193-cmd wants to merge 21 commits into
RyjoxTechnologies:mainfrom
rudi193-cmd:feat/dark-radar-signal
Closed

feat(brain): add DarkRadar fifth signal to BrainHub#1
rudi193-cmd wants to merge 21 commits into
RyjoxTechnologies:mainfrom
rudi193-cmd:feat/dark-radar-signal

Conversation

@rudi193-cmd

Copy link
Copy Markdown

Summary

Adds DarkRadar as a fifth signal to BrainHub in synrix_runtime/monitoring/brain.py.

DarkRadar detects memories that exist in the agent's store but are functionally invisible to semantic search — a silent failure mode where writes succeed but retrievals always miss.

Three detection levels:

  1. None embedding — write was accepted but embedding was never computed
  2. Zero-vector — norm < 1e-6, mathematically invisible to cosine similarity
  3. Search verificationbackend.semantic_search() returns no results for this key (index desync)

Returns a BrainEvent(event_type="dark", severity="warning", action_type="reindex") when triggered. Tracks per-tenant/agent dark counts.

Also adds tests/test_brain.py — 13 unit tests, first coverage for brain.py. All passing.

Why

The existing four signals (LoopBreaker, DriftRadar, ContradictionShield, MemoryHealth) all operate on memories the agent can see. DarkRadar covers the case where the memory is invisible — the agent wrote it, confirmed success, but can never retrieve it. Without this, dark records accumulate silently and degrade agent reasoning quality over time.

Test Plan

  • python3 -m pytest tests/test_brain.py -v — all 13 tests pass
  • process_write() with embedding=None returns a dark event
  • process_write() with zero-vector returns a dark event
  • process_write() with valid embedding and no backend returns None (no false positive)
  • get_brain_status() includes dark_warnings key
  • DarkRadar.reset() clears the count for a tenant/agent pair

Generated with Claude Code

RyjoxTechnologies and others added 21 commits April 2, 2026 19:24
Octopoda: the open-source memory operating system for AI agents.

Core features:
- Persistent memory (SQLite local, PostgreSQL cloud)
- Semantic search (bge-small-en-v1.5, 33MB, CPU-only)
- Loop detection (5-signal intelligence engine)
- Agent-to-agent messaging
- Knowledge graphs (spaCy NER)
- Crash recovery with sub-millisecond restore
- Real-time observability dashboard
- MCP server (25+ tools for Claude/Cursor)

Framework integrations:
- LangChain (BaseMemory + ChatMessageHistory)
- CrewAI (crew memory with task tracking)
- AutoGen (group conversation memory)
- OpenAI Agents SDK (tool functions)

Cloud API:
- 77 REST endpoints (FastAPI)
- Multi-tenant with PostgreSQL RLS
- Stripe billing integration
- Rate limiting per plan

215 tests passing. MIT license.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed 10 unused modules that had zero imports anywhere in the codebase:
- advanced_retrieval_tricks.py, assistant_memory.py, auto_organizer.py
- simplified_hierarchical_organizer.py, hierarchical_organizer.py
- track_growth.py, usage_report.py, feedback.py
- agent_auto_save.py, agent_hooks.py

Fixed README comparison table to be honest about competitors:
- Mem0 is Apache 2.0, not closed source
- Zep has a Community Edition
- Removed "Raw Vector DB" column (not a direct competitor)

Cleaned last json alias in runtime.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Converted 49 print() calls to logger.info/warning/error in 11 files.
Production code now uses Python's logging module consistently.
Only CLI output (cli.py) and docstring examples retain print().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- python-sdk/ was removed in the clean repo, use pip install -e ".[dev]" instead
- Increase health check wait from 15s to 30s (embedding model warmup)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows "Octopoda running locally (SQLite). For cloud sync + dashboard:
https://octopodas.com/signup" once per session when running in local mode.
Only prints once, not per agent.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Critical:
- Dashboard error now tells user: pip install octopoda[server]
- Snapshot add_node() no longer passes unsupported kwargs (_background, metadata)
- Loop detection Signals 2+3 (key overwrites, velocity) now work WITHOUT AI deps
- recall_similar() returns helpful message when embeddings not installed
- consolidate() early returns now include dry_run key

Polish:
- Banner version updated from v1.0.0 to v3.0.3
- Fact extraction logs downgraded from WARNING to DEBUG (no more console flood)
- "No API key" warning fires once per session, not per agent
- test_full_audit.py UTF-8 encoding declaration added (Windows fix)
- Signal 1 tells user to install octopoda[ai] for semantic similarity

43 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extractions were incorrectly scaled per plan (up to 50K on business).
Platform key extractions are 100 for everyone — after that, users
add their own OpenAI/Anthropic key. Also aligned rate limits across
all plan tiers in both billing.py and cloud_server.py.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix agent limit returning 500 instead of 403 (HTTPException was caught
  by generic except in register_agent endpoint)
- Add backend= parameter to all 4 framework integrations (LangChain,
  CrewAI, AutoGen, OpenAI Agents) — enables local-only usage without
  cloud account
- Add _LocalAgentAdapter that wraps SynrixAgentBackend with cloud Agent
  interface (.write/.read/.keys/.search)
- Add missing LangChain methods: store_entity(), get_entity(),
  restore_from_crash()
- Fix v1 integration tests to use local backend (16/16 pass)
- Fix v2 cloud tests to reuse single agent ID (avoids plan limit)
- Mark v2 tests as skip when OCTOPODA_API_KEY not set
- 208 passed, 17 skipped, 0 failed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: /var/lib/octopoda/ directory didn't exist on the server,
_save_verify_codes() silently swallowed the error (bare except: pass),
so codes were emailed but never persisted. Every verify attempt failed
with "Invalid or expired code" — causing users to spam resend.

Fixes:
- Auto-create directory with os.makedirs() before writing
- Log errors instead of silently swallowing them
- Increase code TTL from 10 → 30 minutes (less pressure on users)
- Update email template to say 30 minutes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Point pyproject.toml URLs to Octopoda-OS repo + octopodas.com
- Update test count badge to 208 (actual passing count)
- Replace empty Discord badge with GitHub release badge
- Add PyPI link to footer

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Emails were sent from noreply@octopodas.com but SPF record only exists
for send.octopodas.com. This caused emails to land in spam or be
rejected. Changed default from address to noreply@send.octopodas.com
which has proper SPF, DKIM, and DMARC records via Resend.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dead code with zero imports anywhere in the codebase:
- agent_context_restore.py (366 lines)
- agent_failure_tracker.py (230 lines)
- agent_integration.py (281 lines)
- agent_memory_layer.py (183 lines)
- ai_memory.py (83 lines)
- auto_memory.py (320 lines)
- cursor_integration.py (185 lines)

~1,650 lines of unused code removed. 132 tests still pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Dashboard overview screenshot showing agents, operations, health,
  anomalies, and recovery
- Step-by-step cloud setup instructions (signup, API key, SDK usage)
- Updated pricing table with correct plan limits
- octopoda-login CLI mentioned for interactive setup

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove confusing pip install extras from feature demos (everything
  works out of the box, extras only needed for local-only mode)
- Trim verbose code examples down to essential lines
- Remove redundant comments and explanations
- Keep it scannable — each feature is 3-6 lines, not 15-20

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Dashboard screenshot moved to top (most impressive visual first)
- Quick Start now shows both local and cloud in one section
- Comparison table moved below features (show value before comparing)
- Cloud Dashboard + Cloud Setup merged into one Cloud section
- Architecture section removed (belongs in docs/, not landing page)
- Installation and Config pushed to bottom (reference material)
- Removed redundant "Core Features" header
- Overall shorter and more scannable

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows latency comparison, score breakdown, write/read latency graphs,
and per-agent performance leaderboard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows memory browsing with version history — every change to a memory
is tracked so you can see exactly how agent knowledge evolved over time.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace Alice demo with minimal key/value example
- Add audit trail and dashboard to Why section
- Replace knowledge graphs with more practical features

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Quick Start now shows AgentRuntime alone does everything
- New "What You Get Out of the Box" section explaining what's automatic
- Features section renamed to "When You Need More Control"
- Removed Alice examples, replaced with practical ones
- Comparison table updated (audit trail replaces knowledge graph)
- Friendlier tone throughout

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DarkRadar detects memories that exist in the store but are invisible
to semantic search — a silent data loss worse than not writing at all.

Three detection levels:
1. None embedding — record has no vector, search can never return it
2. Zero-vector embedding — norm < 1e-6, cosine similarity = 0 with
   everything, functionally unfindable
3. Search verification (opt-in, requires backend) — write succeeded
   but backend.semantic_search() returns no match for the written key

Discovered in production: a Willow memory audit found 27/28 atoms (96%)
dark in a Postgres knowledge store due to FTS tokenization gaps and
missing sync between write path and search index. The embedding-based
equivalent happens when the embedding service fails silently — the write
path returns success but the vector never lands in the index.

Wired into BrainHub.process_write() as signal #5.
get_brain_status() now includes dark_warnings count.
get_events() supports filtering by event_type="dark".

13 tests added (tests/test_brain.py) — first test coverage for brain.py.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@RyjoxTechnologies

Copy link
Copy Markdown
Owner

Let me know how you get on

@rudi193-cmd

Copy link
Copy Markdown
Author

Working well on our end. The DarkRadar signal caught a real production failure that motivated the PR — a Postgres knowledge store where 27/28 atoms (96%) were invisible to semantic search due to FTS tokenization gaps. The embeddings existed, the writes confirmed success, but retrievals always missed. Classic silent dark.

The three-tier check (None → zero-vector → live search verification) is the right layering. The search-verification level was the critical one for our use case — embedding-level checks pass, but the records don't surface in actual queries. All 13 tests pass locally.

One thing to note for reviewers: the backend parameter on DarkRadar.check() is optional and defaults to None, so it degrades gracefully in environments without a live backend. That was an intentional design choice to avoid blocking writes.


In case the question was how to wire it in:

from synrix_runtime.monitoring.brain import DarkRadar

# After your memory write returns an embedding:
event = DarkRadar.check(
    tenant_id="your-tenant",
    agent_id="your-agent",
    key=memory_key,
    value=memory_value,
    embedding=embedding_from_write,  # None if write didn't return one
    backend=your_store_backend,       # optional — enables live search verify
)

if event:
    # event.event_type == "dark", event.action_type == "reindex"
    # log it, alert, queue a reindex — your call
    handle_dark_memory(event)

backend is optional — if you pass it, DarkRadar does a live search verification after checking the embedding. If you omit it, you still catch None and zero-vector cases. get_brain_status() on BrainHub now includes a dark_warnings count per tenant/agent.

@rudi193-cmd

Copy link
Copy Markdown
Author

Still working well — the DarkRadar signal has been catching real production failures in our setup (Postgres knowledge store where atoms were invisible to semantic search despite confirmed writes). All 13 tests passing. Let us know if there's anything you'd like adjusted before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants