|
| 1 | +# Experts System |
| 2 | + |
| 3 | +> **Status**: Specification complete, implementation pending. This document describes planned functionality. |
| 4 | +
|
| 5 | +Experts are auto-improving collections of domain knowledge. They provide a structured mechanism for accumulating expertise and exposing it throughout the system. |
| 6 | + |
| 7 | +## Overview |
| 8 | + |
| 9 | +Each expert represents deep knowledge in a specific domain (e.g., "Ruby on Rails ActiveJob", "Social Marketing"). Experts consist of: |
| 10 | +- **Core expertise**: The foundational knowledge about the domain |
| 11 | +- **Topics**: Detailed documentation on specific subjects within the domain |
| 12 | +- **Learnings**: Hard-fought insights from real experiences |
| 13 | + |
| 14 | +## Directory Structure |
| 15 | + |
| 16 | +Experts live in `.deepwork/experts/[expert-folder-name]/`: |
| 17 | + |
| 18 | +``` |
| 19 | +.deepwork/experts/ |
| 20 | +└── rails_activejob/ |
| 21 | + ├── expert.yml # Core expert definition |
| 22 | + ├── topics/ # Detailed topic documentation |
| 23 | + │ └── retry_handling.md |
| 24 | + └── learnings/ # Experience-based insights |
| 25 | + └── job_errors_not_going_to_sentry.md |
| 26 | +``` |
| 27 | + |
| 28 | +### Naming Convention |
| 29 | + |
| 30 | +The **expert name** is derived from the folder name: |
| 31 | +- Spaces and underscores become dashes |
| 32 | +- Example: folder `rails_activejob` → expert name `rails-activejob` |
| 33 | +- This name is used in CLI commands: `deepwork topics --expert "rails-activejob"` |
| 34 | + |
| 35 | +## File Formats |
| 36 | + |
| 37 | +### expert.yml |
| 38 | + |
| 39 | +```yaml |
| 40 | +discovery_description: | |
| 41 | + Short description used by other parts of the system to decide |
| 42 | + whether to invoke this expert. Keep it concise and specific. |
| 43 | +
|
| 44 | +full_expertise: | |
| 45 | + Unlimited text (but generally ~5 pages max) containing the |
| 46 | + complete current knowledge of this domain. This is the core |
| 47 | + expertise that gets included in the generated agent. |
| 48 | +``` |
| 49 | +
|
| 50 | +**Note**: The expert name is not a field in this file—it's derived from the folder name. |
| 51 | +
|
| 52 | +### topics/*.md |
| 53 | +
|
| 54 | +Topics are frontmatter Markdown files covering specific subjects within the domain. |
| 55 | +
|
| 56 | +```markdown |
| 57 | +--- |
| 58 | +name: Retry Handling |
| 59 | +keywords: |
| 60 | + - retry |
| 61 | + - exponential backoff |
| 62 | + - dead letter queue |
| 63 | +last_updated: 2025-01-15 |
| 64 | +--- |
| 65 | + |
| 66 | +Detailed documentation about retry handling in ActiveJob... |
| 67 | +``` |
| 68 | + |
| 69 | +| Field | Description | |
| 70 | +|-------|-------------| |
| 71 | +| `name` | Human-readable name (e.g., "Retry Handling") | |
| 72 | +| `keywords` | Topic-specific keywords only—avoid broad terms like "Rails" | |
| 73 | +| `last_updated` | Date stamp (manually maintained) | |
| 74 | + |
| 75 | +Filenames are purely organizational and don't affect functionality. |
| 76 | + |
| 77 | +### learnings/*.md |
| 78 | + |
| 79 | +Learnings document complex experiences and hard-fought insights—like mini retrospectives. |
| 80 | + |
| 81 | +```markdown |
| 82 | +--- |
| 83 | +name: Job errors not going to Sentry |
| 84 | +last_updated: 2025-01-20 |
| 85 | +summarized_result: | |
| 86 | + Sentry changed their standard gem for hooking into jobs. |
| 87 | + SolidQueue still worked but ActiveJobKubernetes did not. |
| 88 | +--- |
| 89 | + |
| 90 | +## Context |
| 91 | +We noticed errors from background jobs weren't appearing in Sentry... |
| 92 | + |
| 93 | +## Investigation |
| 94 | +After debugging, we discovered that Sentry's latest gem update... |
| 95 | + |
| 96 | +## Resolution |
| 97 | +Updated the initialization to explicitly configure the hook... |
| 98 | +``` |
| 99 | + |
| 100 | +| Field | Description | |
| 101 | +|-------|-------------| |
| 102 | +| `name` | Human-readable title of the learning | |
| 103 | +| `last_updated` | Date stamp (manually maintained) | |
| 104 | +| `summarized_result` | Brief summary of the key finding | |
| 105 | + |
| 106 | +## CLI Commands |
| 107 | + |
| 108 | +### List Topics |
| 109 | + |
| 110 | +```bash |
| 111 | +deepwork topics --expert "rails-activejob" |
| 112 | +``` |
| 113 | + |
| 114 | +Returns a Markdown list of topics: |
| 115 | +- Name and relative file path as a Markdown link |
| 116 | +- Followed by keywords |
| 117 | +- Sorted by most-recently-updated |
| 118 | + |
| 119 | +### List Learnings |
| 120 | + |
| 121 | +```bash |
| 122 | +deepwork learnings --expert "rails-activejob" |
| 123 | +``` |
| 124 | + |
| 125 | +Returns a Markdown list of learnings: |
| 126 | +- Name and relative file path as a Markdown link |
| 127 | +- Followed by the summarized result |
| 128 | +- Sorted by most-recently-updated |
| 129 | + |
| 130 | +## Sync Behavior |
| 131 | + |
| 132 | +`deepwork sync` generates Claude agents from experts (in addition to syncing jobs). |
| 133 | + |
| 134 | +### Generated Agent Location |
| 135 | + |
| 136 | +Agents are created in `.claude/agents/` with: |
| 137 | +- **Filename**: `dwe_[expert-name].md` (e.g., `dwe_rails-activejob.md`) |
| 138 | +- **name field**: `[expert-name]-expert` (e.g., `rails-activejob-expert`) |
| 139 | +- **description**: The `discovery_description` from expert.yml |
| 140 | + |
| 141 | +### Agent Body Content |
| 142 | + |
| 143 | +The agent body combines: |
| 144 | +1. The `full_expertise` text |
| 145 | +2. A topics list using Claude's dynamic command embedding: |
| 146 | + ``` |
| 147 | + $(deepwork topics --expert "rails-activejob") |
| 148 | + ``` |
| 149 | +3. A learnings list using the same mechanism: |
| 150 | + ``` |
| 151 | + $(deepwork learnings --expert "rails-activejob") |
| 152 | + ``` |
| 153 | + |
| 154 | +This ensures the agent always has access to the latest topics and learnings at runtime. |
| 155 | + |
| 156 | +## Standard Experts |
| 157 | + |
| 158 | +Standard experts ship with DeepWork and are located at `src/deepwork/standard/experts/`. |
| 159 | + |
| 160 | +When `deepwork install` runs, these are copied into `.deepwork/experts/` in the target project. |
| 161 | + |
| 162 | +## Future Considerations |
| 163 | + |
| 164 | +- Experts and jobs are currently independent systems |
| 165 | +- Future versions may enable experts to reference each other and collaborate with jobs |
0 commit comments