|
| 1 | +# Architecture |
| 2 | + |
| 3 | +Cortex is a stateful research and retrieval platform built around four cooperating paths: research execution, routed chat, asynchronous resource ingestion, and session-scoped persistence. |
| 4 | + |
| 5 | +## System context |
| 6 | + |
| 7 | +```mermaid |
| 8 | +flowchart TB |
| 9 | + user["Researcher"] |
| 10 | +
|
| 11 | + subgraph cortex["Cortex application"] |
| 12 | + direction TB |
| 13 | + ui["React workspace"] --> api["FastAPI API and SSE streaming"] |
| 14 | + api --> research["LangGraph research workflow"] |
| 15 | + api --> chat["ReAct-lite chat routing"] |
| 16 | + api --> ingestion["Transactional ingestion outbox"] |
| 17 | + end |
| 18 | +
|
| 19 | + user --> ui |
| 20 | + research --> ai["LLM, search, and retrieval providers"] |
| 21 | + chat --> ai |
| 22 | + research --> data["Supabase and Neo4j"] |
| 23 | + chat --> data |
| 24 | + ingestion --> jobs["Inngest workers"] |
| 25 | + jobs --> data |
| 26 | +
|
| 27 | + api -. telemetry .-> observe["LangSmith and LangFuse"] |
| 28 | + jobs -. telemetry .-> observe |
| 29 | +``` |
| 30 | + |
| 31 | +## Research execution |
| 32 | + |
| 33 | +```mermaid |
| 34 | +flowchart LR |
| 35 | + query["User query"] --> api["Research endpoint"] |
| 36 | + api --> search["Search"] |
| 37 | + search -->|"continue"| retrieve["Retrieve and parse"] |
| 38 | + search -->|"abort"| abort["Abort"] |
| 39 | + retrieve -->|"ok"| memory["GraphRAG context"] |
| 40 | + retrieve -->|"empty"| empty["Empty result"] |
| 41 | + memory --> rerank["Rerank"] |
| 42 | + rerank --> summarize["Summarize"] |
| 43 | + summarize --> report["Generate report"] |
| 44 | + report --> stream["SSE stream"] |
| 45 | + stream --> done["End"] |
| 46 | + abort --> done |
| 47 | + empty --> done |
| 48 | +``` |
| 49 | + |
| 50 | +Search, retrieval, memory context, reranking, summarization, and report generation are separate graph nodes. The graph represents empty and aborted execution explicitly instead of collapsing every non-happy path into a generic exception. |
| 51 | + |
| 52 | +## Chat routing |
| 53 | + |
| 54 | +```mermaid |
| 55 | +flowchart LR |
| 56 | + message["User message"] --> context["Available local context"] |
| 57 | + context --> router["Schema-validated ReAct-lite router"] |
| 58 | + router -->|"answer_direct"| direct["Direct answer"] |
| 59 | + router -->|"answer_from_rag"| rag["Grounded RAG answer"] |
| 60 | + router -->|"web_search"| web["Web search"] |
| 61 | + router -->|"fetch_url"| fetch["Fetch URL"] |
| 62 | + router -->|"ask_clarifying"| clarify["Clarifying question"] |
| 63 | +``` |
| 64 | + |
| 65 | +Routing policy: |
| 66 | + |
| 67 | +- Greetings and other social turns normally resolve directly. |
| 68 | +- Weak or empty RAG context does not automatically trigger web search. |
| 69 | +- Web search is selected for external, fresh, or otherwise web-dependent information. |
| 70 | +- A URL in the message or history is available context, not an automatic fetch instruction. |
| 71 | +- Direct URL fetching happens only when inspecting the resource is necessary. |
| 72 | +- Agent chat, workspace chat, and streaming/non-streaming endpoints use the same policy. |
| 73 | + |
| 74 | +Workspace and agent document collections are deny-by-default: Cortex retrieves them only when the router explicitly selects `answer_from_rag`. Session attachments remain available as explicitly scoped resources. A routing failure after one structured-output repair returns `router_error` and does not retrieve documents. |
| 75 | + |
| 76 | +## Reliable ingestion |
| 77 | + |
| 78 | +```mermaid |
| 79 | +flowchart LR |
| 80 | + upload["Upload resource"] --> api["Upload endpoint"] |
| 81 | + api --> tx["Atomic database RPC"] |
| 82 | + tx --> resource["Resource row"] |
| 83 | + tx --> job["Queued ingestion job"] |
| 84 | + tx --> event["Pending outbox event"] |
| 85 | + dispatcher["Outbox dispatcher"] --> claim["Claim event"] |
| 86 | + claim --> publish["Publish to Inngest"] |
| 87 | + publish --> worker["Idempotent ingestion worker"] |
| 88 | + worker --> ingest["Ingest signed storage URL"] |
| 89 | + ingest --> artifacts["GraphRAG artifacts"] |
| 90 | + claim -->|"send error"| retry["Backoff and retry"] |
| 91 | +``` |
| 92 | + |
| 93 | +Resource creation, job creation, and the intent to publish are committed together. The dispatcher claims outbox rows before publishing to Inngest, and the worker claims a queued job before processing it. Retries therefore preserve the event intent without allowing duplicate workers to process the same terminal job. |
| 94 | + |
| 95 | +## Persistence and isolation |
| 96 | + |
| 97 | +- Supabase provides Postgres persistence, authentication, and object storage. |
| 98 | +- Research sessions and retrieved context are scoped to the authenticated user. |
| 99 | +- Neo4j stores document chunks, vector embeddings, and graph relationships used for retrieval. |
| 100 | +- Redis accelerates auth, search, and session hot paths but degrades gracefully when unavailable. |
| 101 | + |
| 102 | +## Key engineering decisions |
| 103 | + |
| 104 | +### Transactional outbox instead of direct dispatch |
| 105 | + |
| 106 | +A direct `database write → queue API call` creates a dual-write failure window: the process can crash after saving the job but before publishing its event. Cortex records the resource, job, and outbox intent in one Supabase transaction. A separate dispatcher publishes and retries independently, avoiding a distributed transaction between Postgres and Inngest. |
| 107 | + |
| 108 | +### Model-directed routing instead of fixed heuristics |
| 109 | + |
| 110 | +Whether a turn needs RAG, the web, a URL fetch, clarification, or no tool at all depends on intent and conversation context. A validated model decision keeps that policy consistent across chat surfaces. The optional fine-tuned router can offload this frequent classification step to a smaller model; see [Router fine-tuning](router-fine-tuning.md). |
| 111 | + |
| 112 | +### Explicit graph outcomes instead of a linear pipeline |
| 113 | + |
| 114 | +An empty search result is not the same as a crashed provider call. Explicit graph edges preserve the distinction so the API, UI, and telemetry can report accurate outcomes and recovery behavior. |
| 115 | + |
| 116 | +## Related documentation |
| 117 | + |
| 118 | +- [Getting started](getting-started.md) |
| 119 | +- [Production deployment](deployment.md) |
| 120 | +- [Observability](observability.md) |
| 121 | +- [Model evaluation](model-evaluation.md) |
| 122 | +- [UI design system](../ui/DESIGN.md) |
0 commit comments