Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
155 changes: 61 additions & 94 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,13 +49,10 @@ This is a **sophisticated multi-agent financial AI platform** with a **hierarchi
```
src/stock_datasource/agents/
├── base_agent.py # LangGraphAgent base class
├── orchestrator.py # OrchestratorAgent - main routing agent
├── orchestrator.py # Lightweight config-driven routing agent
├── config_driven_harness_agent.py # DB-configured Harness agent
├── chat_agent.py # ChatAgent - conversational interface
├── memory_agent.py # MemoryAgent - user memory management
├── workflow_agent.py # WorkflowAgent - workflow execution
├── workflow_generator_agent.py # WorkflowGeneratorAgent - AI workflow generation
├── deep_agent.py # DeepAgent - deep research capability
├── [15 specialized agents] # Market, Report, Portfolio, Index, etc.
├── [specialized agents] # Legacy direct-import compatibility
├── middlewares/ # Safety & memory middleware
│ ├── base.py # Middleware base class
│ ├── cross_validation.py # Cross-validation middleware
Expand All @@ -67,29 +64,26 @@ src/stock_datasource/agents/
└── tools.py # Shared tools
```

### 2.2 The 18+ Specialized Agents
### 2.2 Specialized Agent Names

Most chat-facing agents are now DB configurations executed by `ConfigDrivenHarnessAgent`. Some Python agent files remain only for direct-import compatibility.

| Agent Name | Purpose | Key Responsibilities |
|------------|---------|----------------------|
| **OverviewAgent** | Market overview | Market sentiment, daily trends, aggregate statistics |
| **MarketAgent** | Technical analysis | Stock analysis, price trends, technical indicators |
| **OverviewAgent** | Market overview | DB-configured market sentiment and aggregate analysis |
| **MarketAgent** | Technical analysis | Legacy direct-import compatibility for market tools |
| **ScreenerAgent** | Intelligent stock screening | Find stocks matching criteria, quantitative filtering |
| **ReportAgent** | Financial analysis | Fundamental analysis, financial statements, valuations |
| **ReportAgent** | Financial analysis | Legacy direct-import compatibility for report tools |
| **HKReportAgent** | Hong Kong stocks | HK-specific financial analysis |
| **PortfolioAgent** | Portfolio management | Portfolio monitoring, asset allocation, risk analysis |
| **EnhancedPortfolioAgent** | Advanced portfolio analysis | Deep portfolio diagnostics, optimization |
| **PortfolioAgent** | Portfolio management | Legacy direct-import compatibility for portfolio tools |
| **BacktestAgent** | Strategy backtesting | Historical testing, performance evaluation |
| **IndexAgent** | Index analysis | Index tracking, comparative analysis |
| **EtfAgent** | ETF analysis | ETF holdings, performance, tracking error |
| **TopListAgent** | Limit-up/limit-down boards | Trading sentiment, institutional activity |
| **NewsAnalystAgent** | News analysis | Market news interpretation, sentiment analysis |
| **KnowledgeAgent** | Knowledge retrieval | Research reports, announcements (RAG system) |
| **MemoryAgent** | User memory | Preference storage, portfolio tracking |
| **DataManageAgent** | Data management | Data refresh, quality checks, maintenance |
| **WorkflowAgent** | Multi-step workflows | Complex task orchestration |
| **WorkflowGeneratorAgent** | AI workflow generation | Auto-generate workflow definitions |
| **ChatAgent** | General conversation | Fallback for unmatched queries |
| **DeepAgent** | Deep research | Complex multi-turn analysis |

### 2.3 Arena System (Multi-Agent Competition)

Expand Down Expand Up @@ -124,23 +118,15 @@ src/stock_datasource/arena/

## 3. Core Orchestration Services

### 3.1 Agent Runtime (`services/agent_runtime.py`)
### 3.1 Config-Driven Harness Runtime

**Purpose:** Unified control plane for all agent execution
**Purpose:** Build and execute agents from ClickHouse `agent_configs` records.

**Architecture:**
- Uses **LangGraph Supervisor** (`langgraph_supervisor.create_supervisor`)
- Native multi-agent patterns with `create_react_agent`
- Handoff mechanism via `Command(goto=...)`
- **v2 streaming** with `astream_events(version="v2")`

**Key Features:**
- SSE event adaptation for frontend compatibility
- Observability metrics (cold start, token cost, classification count)
- Middleware chain integration
- Memory store for cross-session persistence

**Feature Flag:** `AGENT_RUNTIME_ENABLED` env var
- `OrchestratorAgent` performs lightweight intent classification.
- `ConfigDrivenHarnessAgent` loads the selected DB config and builds a DeepAgents harness.
- `tool_registry.py` resolves configured skills/tool names to callable functions.
- No `AGENT_RUNTIME_ENABLED` or `HARNESS_MODE_ENABLED` switch is required.

### 3.2 Execution Planner (`services/execution_planner.py`)

Expand Down Expand Up @@ -326,21 +312,21 @@ Competition Phase
└── Periodic elimination & replenishment
```

### 5.3 Workflow Agent Pattern (WorkflowAgent)
### 5.3 Config-Driven Agent Pattern

**Flow:**
```
Workflow Definition (YAML/JSON)
├── Steps (each step → agent call)
├── Conditional logic
├── Parallel execution groups
└── Error handling
Agent config (ClickHouse)
├── system_prompt
├── skills / tool names
├── model_config
└── runtime_config
WorkflowAgent executes steps
├── Sequential or parallel execution
├── State management between steps
├── Tool invocation
└── Fallback on error
ConfigDrivenHarnessAgent executes request
├── Resolve tools via tool_registry
├── Build create_deep_agent harness
├── Stream SSE-compatible events
└── Persist session metadata
```

### 5.4 Middleware Chain (Safety & Memory)
Expand Down Expand Up @@ -544,11 +530,11 @@ Each module provides:
- Features: Discussion modes, backtesting, elimination
- Specs affected: `multi-agent-arena`, `strategy-competition`, `agent-discussion`

**Refactor Agent Runtime Extensibility** (`refactor-agent-runtime-extensibility/`)
**Orchestrator Simplification** (`refactor-orchestrator-simplify/`)
- Status: Active proposal
- Scope: Unified orchestration layer
- Goals: Converge OrchestratorAgent, WorkflowAgent, MultiAgentArena into single runtime
- Key changes: Agent Registry, Execution Planner, Session Memory Service
- Scope: Config-driven Harness as the chat orchestration path
- Goals: Remove feature-flagged runtime layers and deprecated agent files
- Key changes: Lightweight OrchestratorAgent, ConfigDrivenHarnessAgent dispatch, tool registry resolution

**Intelligent Strategy System** (`add-intelligent-strategy-system/`)
- Status: Archived 2026-01-11
Expand All @@ -569,39 +555,27 @@ openspec/specs/

## 11. Hierarchical Execution Patterns

### 11.1 Three-Tier Hierarchy
### 11.1 Two-Tier Chat Hierarchy

**Tier 1: OrchestratorAgent**
- Top-level router
- Top-level chat router
- Intent classification
- Agent selection
- Concurrent/sequential execution coordination
- Middleware chain orchestration

**Tier 2: Specialized Agents**
- Domain-specific agents (MarketAgent, ReportAgent, etc.)
- Tool invocation
- LLM reasoning
- Result formatting

**Tier 3: WorkflowAgent / MultiAgentArena**
- Multi-step orchestration
- Sub-agent coordination
- Complex reasoning
- Arena-specific competition logic
- Config-driven agent selection
- SSE metadata normalization

**Tier 2: ConfigDrivenHarnessAgent**
- Loads DB agent configuration
- Resolves tools through `tool_registry`
- Builds the DeepAgents harness
- Streams tool/content/done events

### 11.2 Agent Discovery & Loading

**Process:**
1. Runtime scans `src/stock_datasource/agents/` for `*_agent.py` files
2. Extracts classes inheriting from `LangGraphAgent`
3. Creates AgentDescriptor entries
4. Lazy instantiation on demand
5. Caching for performance

**Classes Excluded:**
- `OrchestratorAgent` (special role)
- `StockDeepAgent` (deprecated)
1. Orchestrator reads visible agent configs from ClickHouse `agent_configs`.
2. LLM classification selects an `agent_name` from that catalog.
3. `get_config_driven_agent(agent_name)` loads and caches the harness agent.
4. Skills/tool names resolve through `tool_registry`.

---

Expand Down Expand Up @@ -635,10 +609,7 @@ Centralized discovery:
- Plugin registry

### 12.5 Adapter Pattern
WorkflowAgent and MultiAgentArena are adapters that:
- Convert domain-specific requests to agent calls
- Provide specialized orchestration
- Integrate with main runtime
External systems can adapt domain-specific requests by creating or selecting DB agent configurations and invoking the config-driven harness path.

---

Expand Down Expand Up @@ -715,24 +686,20 @@ OrchestratorAgent
## 15. Summary: Multi-Agent Architecture Highlights

### Strengths:
1. **Modular Design**: 18+ specialized agents with clear responsibilities
2. **Flexible Orchestration**: Multiple coordination patterns (route-only, parallel, handoff, discussion)
3. **Safety & Control**: 5 middleware layers for validation, memory, safety
4. **Scalability**: Concurrent agent execution, async/await throughout
5. **Observability**: Langfuse tracing, SSE event streaming, debug metadata
6. **Extensibility**: Plugin system, skill registry, strategy registry
7. **Arena Competition**: Unique multi-agent debate system for strategy refinement
8. **Unified Runtime**: Converging toward single control plane (in progress)
1. **Config-Driven Design**: Chat-facing agents are managed through DB configuration.
2. **Simplified Orchestration**: Orchestrator performs classification and delegates to one harness path.
3. **Tool Registry**: Skills map consistently to callable tool functions.
4. **Observability**: Langfuse tracing, SSE event streaming, debug metadata.
5. **Extensibility**: Plugin system, skill registry, strategy registry.
6. **Arena Competition**: Separate multi-agent debate system for strategy refinement.

### Complexity Points:
1. Multiple agent discovery mechanisms (registry + scanning)
2. Complex state management (session, cache, memory layers)
3. Heavy middleware chain (5 transforms per request)
4. Arena state machine (6 states, concurrent management)
5. Triple execution modes (OrchestratorAgent, WorkflowAgent, MultiAgentArena)
1. DB agent catalog availability now determines chat routing coverage.
2. Some legacy direct-import agents remain for compatibility.
3. Arena state machine remains separate from the lightweight chat orchestrator.

### Future Evolution (Planned):
1. Complete Agent Runtime unification (Phase 1 in progress)
1. Continue config-driven Harness consolidation
2. Explicit Skill Registry standardization
3. SubAgent protocol for better sub-task isolation
4. High-cost path optimization (reduce redundant classification/context)
Expand All @@ -744,12 +711,12 @@ OrchestratorAgent

**Key Agent Files:**
- `src/stock_datasource/agents/base_agent.py` - LangGraphAgent interface
- `src/stock_datasource/agents/orchestrator.py` - Main orchestrator (500+ lines)
- `src/stock_datasource/agents/memory_agent.py` - User memory management
- `src/stock_datasource/agents/workflow_agent.py` - Workflow execution
- `src/stock_datasource/agents/orchestrator.py` - Lightweight config-driven orchestrator
- `src/stock_datasource/agents/config_driven_harness_agent.py` - DB-configured Harness agent
- `src/stock_datasource/agents/tools.py` - Shared tool functions

**Key Service Files:**
- `src/stock_datasource/services/agent_runtime.py` - Unified control plane
- `src/stock_datasource/services/tool_registry.py` - Skill/tool resolution
- `src/stock_datasource/services/execution_planner.py` - Routing config
- `src/stock_datasource/services/agent_registry.py` - Agent discovery
- `src/stock_datasource/services/agent_cache.py` - Shared caching
Expand Down Expand Up @@ -783,5 +750,5 @@ This is a **production-grade multi-agent system** with sophisticated orchestrati
- **Services** (infrastructure for runtime, registry, memory)
- **Middleware** (cross-cutting concerns like safety and memory)

The system is actively evolving toward a unified `Agent Runtime` that will consolidate three separate orchestration layers (OrchestratorAgent, WorkflowAgent, MultiAgentArena) into a single control plane, following patterns from advanced systems like OpenClaw.
The chat orchestration path now favors a lightweight `OrchestratorAgent` plus `ConfigDrivenHarnessAgent`, with legacy direct-import agents retained only where external modules still depend on them.

2 changes: 1 addition & 1 deletion docs/HARNESS_MIGRATION_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,4 +214,4 @@ Phase 3 (1天): P2 + 回归测试
| SubAgentMiddleware 不支持自定义 SSE event 格式 | 保留 execute_stream wrapper,只替内部逻辑 |
| MemoryMiddleware 与现有 FactExtractor 冲突 | 两者并行运行,MemoryMiddleware 管会话级,FactExtractor 管知识级 |
| deepagents 版本不稳定 | pin 版本,封装 adapter 层 |
| 迁移期间功能回退 | feature flag `HARNESS_MODE_ENABLED` 控制新旧路径 |
| 迁移期间功能回退 | 历史方案曾使用 feature flag;当前默认走 `ConfigDrivenHarnessAgent` |
24 changes: 9 additions & 15 deletions docs/HARNESS_MIGRATION_METHOD.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Harness 模式为每个 Agent 增加以下能力:
- **InMemoryStore**: 跨会话持久化存储
- **astream_events v2**: 与现有 SSE 管道完全兼容

迁移目标:在**不修改原 Agent 任何代码**的前提下,创建一个 Harness 变体,通过环境变量 `HARNESS_MODE_ENABLED=true` 切换。
迁移目标(历史):在**不修改原 Agent 任何代码**的前提下创建 Harness 变体。当前主链路已改为默认使用 `ConfigDrivenHarnessAgent`,不再通过 `HARNESS_MODE_ENABLED` 切换。

---

Expand Down Expand Up @@ -78,8 +78,7 @@ def _get_harness_store() -> InMemoryStore:
_harness_store = InMemoryStore()
return _harness_store

def is_harness_mode_enabled() -> bool:
return os.getenv("HARNESS_MODE_ENABLED", "").lower() == "true"
# 历史方案曾使用 HARNESS_MODE_ENABLED 开关;当前不再需要 feature flag。
```

### Step 5: 实现 `_init_harness_agent`
Expand Down Expand Up @@ -130,17 +129,14 @@ def _init_harness_agent(self):
6. 处理 `on_tool_start`、`on_tool_end`、`on_chat_model_stream` 事件
7. 保存历史、发送 agent_end + done 事件

### Step 7: Orchestrator 添加 Feature Flag
### Step 7: 接入配置驱动 Orchestrator

`orchestrator.py` 的单 Agent 路由逻辑中添加:
当前 `orchestrator.py` 默认通过 `get_config_driven_agent(agent_name)` 调度,无需新增 feature flag。

```python
if plan[0] == "OriginalAgent":
from .harness_original_agent import is_harness_mode_enabled
if is_harness_mode_enabled():
from .harness_original_agent import get_harness_original_agent
agent = get_harness_original_agent()
logger.info("[Harness] Using Harness<Agent> instead of <Agent>")
from .config_driven_harness_agent import get_config_driven_agent

agent = get_config_driven_agent(agent_name)
```

### Step 8: 添加 Singleton 工厂
Expand All @@ -161,8 +157,8 @@ def get_harness_xxx_agent() -> HarnessXxxAgent:
# 1. 导入测试
python -c "from stock_datasource.agents.harness_xxx_agent import HarnessXxxAgent; a = HarnessXxxAgent(); print(f'OK: {len(a.get_tools())} tools')"

# 2. 功能测试(需要设置环境变量
HARNESS_MODE_ENABLED=true python -c "..."
# 2. 功能测试(ConfigDrivenHarnessAgent 已是默认路径
python -c "..."
```

---
Expand All @@ -171,7 +167,6 @@ HARNESS_MODE_ENABLED=true python -c "..."

| 组件 | 说明 |
|------|------|
| `is_harness_mode_enabled()` | 环境变量检查 |
| `_harness_store` 单例 | `InMemoryStore` 模块级单例 |
| `_get_harness_store()` | Store 获取函数 |
| `AgentConfig(name=..., description=..., temperature=0.5, max_tokens=8000)` | 配置 |
Expand Down Expand Up @@ -207,7 +202,6 @@ HARNESS_MODE_ENABLED=true python -c "..."
| `_harness_agent` 未初始化 | `__init__` 中设为 `None`,首次调用时初始化 |
| 工具函数签名不兼容 | 确保工具有 docstring + type hints(LangGraph 需要) |
| SSE 事件格式不一致 | 严格复用模板中的 event dict 结构 |
| 环境变量大小写 | `is_harness_mode_enabled()` 已做 `.lower()` |

---

Expand Down
23 changes: 23 additions & 0 deletions openspec/changes/refactor-orchestrator-simplify/proposal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Change: 精简 Orchestrator 为配置驱动 Harness 路径

## Why

`OrchestratorAgent` 当前保留旧 agent 扫描、AgentRuntime feature flag、Harness feature flag、MCP fallback、多 agent/arena 队列等多套执行路径,导致聊天主链路难以维护。`ConfigDrivenHarnessAgent` 已可从 DB 配置动态构建 agent,并通过 `tool_registry` 解析 skill/tool,具备成为默认路径的条件。

## What Changes

- 移除 `HARNESS_MODE_ENABLED` 与 `AGENT_RUNTIME_ENABLED` 双路径开关。
- 删除 `services/agent_runtime.py` 中间层,聊天调度直接进入 `ConfigDrivenHarnessAgent`。
- 精简 `agents/orchestrator.py`,仅保留意图分类、股票代码提取、配置驱动 agent 调度和 SSE 兼容包装。
- 删除无保留价值的废弃 agent 文件,并修正仍指向这些文件的兼容引用。
- 保留 `market_agent.py`、`report_agent.py`、`portfolio_agent.py` 的直接 import 兼容,但标记 deprecated。

## Impact

- Affected specs: `chat-orchestration`
- Affected code:
- `src/stock_datasource/agents/orchestrator.py`
- `src/stock_datasource/agents/config_driven_harness_agent.py`
- `src/stock_datasource/services/agent_runtime.py`
- deprecated files under `src/stock_datasource/agents/`
- workflow/overview/daily-analysis references and related tests/docs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
## MODIFIED Requirements

### Requirement: Chat协调Agent调度
系统 MUST 在chat入口使用协调Agent进行意图解析,并 MUST 通过数据库中的 Agent 配置构建 `ConfigDrivenHarnessAgent` 执行请求。

#### Scenario: 用户发起对话请求
- **WHEN** 用户向chat入口发送消息
- **THEN** 协调Agent解析意图并选择合适的配置驱动Agent执行

#### Scenario: 配置驱动Agent缺失
- **WHEN** 协调Agent无法找到匹配的数据库Agent配置
- **THEN** 系统返回可理解的错误事件而不是回退到旧Agent文件或AgentRuntime

### Requirement: Agent发现与能力编目
系统 MUST 从数据库 Agent 配置中读取可用Agent清单与能力描述以供调度。

#### Scenario: 系统启动或首次请求
- **WHEN** chat入口初始化调度
- **THEN** 协调Agent获取数据库中可见的Agent清单与能力描述

### Requirement: MCP回退调度
系统 MUST NOT 在chat协调入口维护独立的MCP回退执行路径;工具能力 MUST 通过配置驱动Agent的 skills 与 tool registry 装配。

#### Scenario: 无匹配Agent
- **WHEN** 协调Agent无法匹配可用Agent
- **THEN** 系统返回标准 error/done 事件并提示缺少可用Agent配置
9 changes: 9 additions & 0 deletions openspec/changes/refactor-orchestrator-simplify/tasks.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
## 1. Implementation

- [x] 1.1 Create `refactor/orchestrator-simplify` branch from clean `main`.
- [x] 1.2 Rewrite `orchestrator.py` to use config-driven harness dispatch only.
- [x] 1.3 Remove `HARNESS_MODE_ENABLED` and `AGENT_RUNTIME_ENABLED` code paths.
- [x] 1.4 Delete `agent_runtime.py` and deprecated agent files.
- [x] 1.5 Update imports/routes/tests/docs that referenced removed files.
- [x] 1.6 Mark retained legacy direct-import agents as deprecated.
- [x] 1.7 Run reference search, compile checks, and targeted tests.
Loading
Loading