Skip to content

Commit 34861bf

Browse files
thatsmeclaude
andcommitted
Add OTP circuit breaker per skill, workflow resilience controls, UI tooltips (v0.2.3)
Pure OTP circuit breaker (GenServer + ETS) wraps skill execution transparently. After 3 consecutive failures the circuit opens and calls are rejected instantly. Auto-recovery via half-open probe after 5 minutes. Telegram notifications on state transitions. Logs page gets dedicated Circuit Breaker filter (blue badge). Workflow steps gain per-step resilience: on_circuit_open (halt/skip/fallback), on_missing_skill (halt/skip), and fallback_skill — all configurable via new dropdowns in the workflow editor UI. Every form field now has a ? tooltip. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 40180e1 commit 34861bf

14 files changed

Lines changed: 692 additions & 73 deletions

File tree

ALEXCLAW_ARCHITECTURE.md

Lines changed: 42 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@ AlexClaw.Application (one_for_one)
2828
├── AlexClaw.LogBuffer # In-memory ring buffer for recent logs (500 entries)
2929
├── AlexClaw.Google.TokenManager # Google OAuth2 token lifecycle (ETS cache + auto-refresh)
3030
├── AlexClaw.RateLimiter.Server # ETS owner for login rate limiting + periodic purge
31+
├── Registry (CircuitBreakerRegistry) # Process registry for per-skill circuit breakers
32+
├── AlexClaw.Skills.CircuitBreakerSupervisor # DynamicSupervisor — per-skill circuit breakers
3133
├── AlexClaw.SkillSupervisor # DynamicSupervisor — spawns skill worker processes
3234
├── AlexClaw.Scheduler # Quantum cron scheduler
3335
├── AlexClaw.Workflows.SchedulerSync # Syncs DB workflow schedules into Quantum jobs
@@ -125,7 +127,33 @@ Categories: `identity`, `llm`, `embedding`, `telegram`, `skills`, `github`, `goo
125127

126128
### LogBuffer — `AlexClaw.LogBuffer`
127129

128-
In-memory ring buffer (500 entries max) attached to the Erlang Logger. Classifies log entries by severity (`critical`, `high`, `moderate`, `low`) using pattern matching on log level and message content. Filters out verbose OTP/Ecto noise. Exposes `recent/1` for filtered retrieval and `counts/0` for severity aggregates. Powers the real-time log viewer in the admin UI.
130+
In-memory ring buffer (500 entries max) attached to the Erlang Logger. Classifies log entries by severity (`critical`, `high`, `moderate`, `low`, `circuit_breaker`) using pattern matching on log level and message content. Circuit breaker events are identified by the `[CircuitBreaker]` prefix. Filters out verbose OTP/Ecto noise. Exposes `recent/1` for filtered retrieval and `counts/0` for severity aggregates. Powers the real-time log viewer in the admin UI.
131+
132+
### Circuit Breaker — `AlexClaw.Skills.CircuitBreaker`
133+
134+
Per-skill OTP circuit breaker. One GenServer per skill, managed by `CircuitBreakerSupervisor` (DynamicSupervisor). Uses ETS (`:circuit_breakers`) for lock-free state reads on the hot path and `Process.send_after` for reset timers. Zero external dependencies.
135+
136+
**States:** `:closed` (normal) → `:open` (failing, calls rejected) → `:half_open` (testing with one probe call)
137+
138+
**Behavior:**
139+
- Skill fails 3 consecutive times → circuit opens → Telegram notification
140+
- Circuit open → `{:error, :circuit_open}` returned instantly, skill not called
141+
- After 5 minutes → half-open → one test call allowed
142+
- Test succeeds → circuit closes (recovered) → Telegram notification
143+
- Test fails → circuit reopens → wait again
144+
145+
**Integration:** `CircuitBreaker.call/2` wraps skill execution transparently in the Executor. Skills have zero awareness of the breaker — no changes to `run/1` signatures or args.
146+
147+
**Workflow resilience:** Per-step config controls behavior when a circuit is open or skill is missing:
148+
- `on_circuit_open`: `"halt"` (default) | `"skip"` (pass input through) | `"fallback"` (route to alternative skill)
149+
- `on_missing_skill`: `"halt"` (default) | `"skip"` (pass input through)
150+
- `fallback_skill`: name of the alternative skill (used with `"fallback"`)
151+
152+
These are configurable per step via the workflow editor UI (dropdowns, not JSON).
153+
154+
**Dynamic skill lifecycle:** Subscribes to PubSub `"skills:registry"`. When a dynamic skill is unloaded, its breaker is cleaned up. When reloaded, the breaker resets (fresh code = fresh circuit).
155+
156+
**Observability:** Circuit breaker events are classified as `:circuit_breaker` severity in `LogBuffer` (matched by `[CircuitBreaker]` prefix). The Logs page has a dedicated "Circuit Breaker" filter with blue badge.
129157

130158
### Rate Limiter — `AlexClaw.RateLimiter`
131159

@@ -197,6 +225,15 @@ LLM provider selection can be configured at three levels (most specific wins):
197225

198226
All workflow executions run under `AlexClaw.TaskSupervisor`. Run history with step-level results (output, duration, success/failure) is stored in the database and visible in the admin UI.
199227

228+
### Resilience
229+
230+
Each workflow step has per-step resilience controls configured via the workflow editor UI:
231+
- **On Circuit Open** — what to do when the skill's circuit breaker is open: halt the workflow, skip the step (pass input through to the next step), or route to a fallback skill
232+
- **On Missing Skill** — what to do when the skill is not loaded (e.g. dynamic skill unloaded): halt or skip
233+
- **Fallback Skill** — alternative skill to execute when "Fallback skill" is selected
234+
235+
The circuit breaker wraps skill execution transparently in `Executor.execute_step/5`. Skills are unaware of the breaker.
236+
200237
### Notifications
201238

202239
When a workflow contains a `telegram_notify` step, the executor sends a start notification when the workflow begins and a failure notification if any step fails before reaching the notify step.
@@ -286,7 +323,7 @@ Phoenix LiveView admin UI. Session-based authentication — all routes except `/
286323
| Memory | `/memory` | Browse and search stored knowledge |
287324
| Database | `/database` | Schema browser and backup download |
288325
| Config | `/config` | Runtime configuration editor (collapsible categories) |
289-
| Logs | `/logs` | Real-time log viewer with severity filtering |
326+
| Logs | `/logs` | Real-time log viewer with severity filtering (includes circuit breaker events) |
290327

291328
---
292329

@@ -316,6 +353,8 @@ lib/
316353
resource.ex # Resource Ecto schema
317354
migrator.ex # Resource migration utilities
318355
skills/
356+
circuit_breaker.ex # Per-skill OTP circuit breaker (GenServer + ETS)
357+
circuit_breaker_supervisor.ex # DynamicSupervisor + PubSub lifecycle
319358
api_request.ex # HTTP requests with retries
320359
conversational.ex # Free-text LLM conversation
321360
github_security_review.ex # PR/commit security analysis
@@ -337,7 +376,7 @@ lib/
337376
workflow_resource.ex # Join schema (workflow ↔ resource)
338377
workflow_run.ex # Run history Ecto schema
339378
workflow_step.ex # Step Ecto schema (order, config, input_from)
340-
application.ex # Supervision tree (13 children)
379+
application.ex # Supervision tree (15 children)
341380
dispatcher.ex # Telegram command routing (pattern matching)
342381
gateway.ex # Telegram bot (long-polling GenServer)
343382
identity.ex # Persona / system prompt builder

README.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,10 @@ AlexClaw monitors the world (RSS feeds, web sources, GitHub repositories, APIs),
1717
### Core
1818

1919
- **Multi-Model LLM Router** — Tier-based routing (`light` / `medium` / `heavy` / `local`) with priority-based selection. All providers (cloud and local) are stored in PostgreSQL and fully manageable from the admin UI. Tracks daily usage per provider in ETS. Ships with default providers (Gemini, Claude, Ollama, LM Studio) seeded on first boot — add, remove, or reconfigure any provider at runtime.
20-
- **Workflow Engine** — Define multi-step pipelines combining skills and LLM transforms. Each step passes output to the next. Runs on schedule (cron) or on demand. Full run history with step-level results in the admin UI.
20+
- **Workflow Engine** — Define multi-step pipelines combining skills and LLM transforms. Each step passes output to the next. Runs on schedule (cron) or on demand. Full run history with step-level results in the admin UI. Per-step resilience controls (circuit breaker, missing skill handling, fallback routing).
21+
- **OTP Circuit Breaker** — Per-skill circuit breaker using GenServer + ETS. After consecutive failures a skill is temporarily disabled (circuit open), then automatically re-tested after a cooldown. Telegram notifications on state transitions. Dead letter routing: workflow steps can skip, halt, or fallback to an alternative skill when a circuit is open or a skill is missing. Zero external dependencies — pure OTP.
22+
23+
![AlexClaw Circuit Breaker](docs/screenshot/circuit_break.jpg)
2124
- **Telegram Gateway** — Bidirectional communication via long-polling. Command routing is deterministic pattern-matching — no LLM involved in dispatch.
2225
- **Runtime Configuration** — All settings (API keys, prompts, limits, personas) are stored in PostgreSQL, cached in ETS, and editable at runtime via the admin UI. No restart required for any config change.
2326
- **Persistent Memory with Semantic Search** — PostgreSQL + pgvector for knowledge storage. Deduplication by URL. Hybrid search combines vector cosine similarity and keyword matching — vector results are prioritized, keyword results fill gaps for exact matches. Embeddings are generated asynchronously via the LLM router (Gemini `gemini-embedding-001`, Ollama `nomic-embed-text`, or any OpenAI-compatible endpoint). 768-dimension vectors with HNSW index. All skills that store knowledge auto-embed in the background.
@@ -117,11 +120,11 @@ Admin UI (Chat) ──────> SkillSupervisor ──> Dynamic Skills
117120
↑ semantic search ↑
118121
119122
GitHub Webhook ──> WebhookController ──> GitHubSecurityReview
120-
Scheduler (Quantum) ──> Workflows.Executor ──> Skills
121-
Phoenix LiveView Admin ──> all of the above
123+
Scheduler (Quantum) ──> Workflows.Executor ──┬──> CircuitBreaker ──> Skills
124+
Phoenix LiveView Admin ──> all of the above └──> Fallback / Skip / Halt
122125
```
123126

124-
Every skill runs as an isolated OTP process. Crashes are contained and supervised. The `Dispatcher` is deterministic pattern-matching — no LLM token cost for routing.
127+
Every skill runs as an isolated OTP process. Crashes are contained and supervised. The circuit breaker wraps each skill transparently — skills have zero awareness of it. The `Dispatcher` is deterministic pattern-matching — no LLM token cost for routing.
125128

126129
See [ALEXCLAW_ARCHITECTURE.md](ALEXCLAW_ARCHITECTURE.md) for the full design document.
127130

@@ -252,7 +255,7 @@ lib/
252255
config/ # Runtime config (DB + ETS + PubSub broadcast)
253256
llm/ # LLM router, usage tracker, provider schema
254257
memory/ # Memory entry schema
255-
skills/ # Core skill modules, SkillAPI, DynamicSkill schema
258+
skills/ # Core skill modules, SkillAPI, DynamicSkill schema, CircuitBreaker
256259
workflows/ # Executor, scheduler sync, SkillRegistry (GenServer+ETS), step/run schemas
257260
dispatcher.ex # Deterministic message routing
258261
gateway.ex # Telegram bot

SECURITY.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,13 @@ The following protections are in place:
9999
- **Core protection** — core skills cannot be unloaded or overwritten by dynamic skills
100100
- **No NIF compilation** — the Alpine runtime image has no build tools, preventing native code loading
101101

102+
**Circuit breaker protection:** Each skill (core and dynamic) is wrapped by an
103+
OTP circuit breaker. After 3 consecutive failures, the circuit opens and calls
104+
are rejected instantly without executing the skill. This prevents a failing
105+
dynamic skill from consuming resources or cascading failures through workflows.
106+
Workflow steps can be configured to skip or fallback to an alternative skill
107+
when a circuit is open or a skill is missing.
108+
102109
**What is NOT sandboxed:** A dynamic skill runs in the same BEAM VM as the rest
103110
of AlexClaw. A malicious skill could bypass SkillAPI by calling internal modules
104111
directly. The permission system is a guardrail, not a security boundary.

docs/screenshot/circuit_break.jpg

106 KB
Loading

lib/alex_claw/application.ex

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@ defmodule AlexClaw.Application do
1414
AlexClaw.LogBuffer,
1515
AlexClaw.Google.TokenManager,
1616
AlexClaw.RateLimiter.Server,
17+
{Registry, keys: :unique, name: AlexClaw.CircuitBreakerRegistry},
18+
AlexClaw.Skills.CircuitBreakerSupervisor,
1719
AlexClaw.SkillSupervisor,
1820
AlexClaw.Scheduler,
1921
AlexClaw.Workflows.SchedulerSync,

lib/alex_claw/log_buffer.ex

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ defmodule AlexClaw.LogBuffer do
5353
end
5454

5555
@doc "Return counts per severity."
56-
@spec counts() :: %{critical: non_neg_integer(), high: non_neg_integer(), moderate: non_neg_integer(), low: non_neg_integer()}
56+
@spec counts() :: %{critical: non_neg_integer(), high: non_neg_integer(), moderate: non_neg_integer(), low: non_neg_integer(), circuit_breaker: non_neg_integer()}
5757
def counts do
5858
entries =
5959
@table
@@ -65,7 +65,8 @@ defmodule AlexClaw.LogBuffer do
6565
critical: Enum.count(entries, &(&1 == :critical)),
6666
high: Enum.count(entries, &(&1 == :high)),
6767
moderate: Enum.count(entries, &(&1 == :moderate)),
68-
low: Enum.count(entries, &(&1 == :low))
68+
low: Enum.count(entries, &(&1 == :low)),
69+
circuit_breaker: Enum.count(entries, &(&1 == :circuit_breaker))
6970
}
7071
end
7172

@@ -136,6 +137,7 @@ defmodule AlexClaw.LogBuffer do
136137

137138
defp classify(level, message) do
138139
cond do
140+
circuit_breaker_pattern?(message) -> :circuit_breaker
139141
level == :emergency or level == :alert -> :critical
140142
level == :critical -> :critical
141143
level == :error and critical_pattern?(message) -> :critical
@@ -146,6 +148,10 @@ defmodule AlexClaw.LogBuffer do
146148
end
147149
end
148150

151+
defp circuit_breaker_pattern?(msg) do
152+
String.contains?(msg, "[CircuitBreaker]")
153+
end
154+
149155
defp critical_pattern?(msg) do
150156
String.contains?(msg, "not configured") or
151157
String.contains?(msg, "connection refused") or
Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
defmodule AlexClaw.Skills.CircuitBreaker do
2+
@moduledoc """
3+
OTP circuit breaker per skill. Wraps skill execution transparently —
4+
skills have zero awareness of the breaker.
5+
6+
States: :closed (normal) → :open (failing) → :half_open (testing)
7+
8+
Uses ETS for lock-free reads on the hot path and Process.send_after
9+
for reset timers. One GenServer per skill, managed by CircuitBreakerSupervisor.
10+
"""
11+
use GenServer
12+
require Logger
13+
14+
@ets_table :circuit_breakers
15+
@max_failures Application.compile_env(:alex_claw, [:circuit_breaker, :max_failures], 3)
16+
@reset_timeout Application.compile_env(:alex_claw, [:circuit_breaker, :reset_timeout], :timer.minutes(5))
17+
18+
# --- Client API ---
19+
20+
@doc "Start a circuit breaker GenServer for a skill."
21+
@spec start_link(String.t()) :: GenServer.on_start()
22+
def start_link(skill_name) do
23+
GenServer.start_link(__MODULE__, skill_name, name: via(skill_name))
24+
end
25+
26+
@doc """
27+
Transparent wrapper for skill execution. Checks circuit state,
28+
executes the function if allowed, records the result.
29+
Skills see no difference — the breaker is invisible.
30+
"""
31+
@spec call(String.t(), (-> {:ok, any()} | {:error, any()})) ::
32+
{:ok, any()} | {:error, any()} | {:error, :circuit_open}
33+
def call(skill_name, fun) do
34+
AlexClaw.Skills.CircuitBreakerSupervisor.ensure_started(skill_name)
35+
36+
if allow?(skill_name) do
37+
case fun.() do
38+
{:ok, result} ->
39+
record_success(skill_name)
40+
{:ok, result}
41+
42+
{:error, reason} ->
43+
record_failure(skill_name, reason)
44+
{:error, reason}
45+
end
46+
else
47+
{:error, :circuit_open}
48+
end
49+
end
50+
51+
@doc "Check if a skill is allowed to execute. Pure ETS read, no GenServer call."
52+
@spec allow?(String.t()) :: boolean()
53+
def allow?(skill_name) do
54+
case :ets.lookup(@ets_table, skill_name) do
55+
[{_, :open, _, _, _}] -> false
56+
_ -> true
57+
end
58+
end
59+
60+
@doc "Return the circuit state for a skill."
61+
@spec state(String.t()) :: {atom(), non_neg_integer(), any()} | :unknown
62+
def state(skill_name) do
63+
case :ets.lookup(@ets_table, skill_name) do
64+
[{_, state, count, last_error, _}] -> {state, count, last_error}
65+
[] -> :unknown
66+
end
67+
end
68+
69+
@doc "Force-reset a circuit to :closed. Used when a dynamic skill is reloaded."
70+
@spec reset(String.t()) :: :ok
71+
def reset(skill_name) do
72+
case Registry.lookup(AlexClaw.CircuitBreakerRegistry, skill_name) do
73+
[{pid, _}] -> GenServer.call(pid, :reset)
74+
[] -> :ok
75+
end
76+
end
77+
78+
# --- GenServer callbacks ---
79+
80+
@impl true
81+
def init(skill_name) do
82+
:ets.insert(@ets_table, {skill_name, :closed, 0, nil, DateTime.utc_now()})
83+
{:ok, %{skill_name: skill_name, timer_ref: nil}}
84+
end
85+
86+
@impl true
87+
def handle_cast({:record_failure, reason}, state) do
88+
{_, current_state, count, _, _} = lookup!(state.skill_name)
89+
new_count = count + 1
90+
91+
case current_state do
92+
:closed when new_count >= @max_failures ->
93+
transition(state, :open, new_count, reason)
94+
95+
:half_open ->
96+
transition(state, :open, new_count, reason)
97+
98+
_ ->
99+
:ets.insert(@ets_table, {state.skill_name, current_state, new_count, reason, DateTime.utc_now()})
100+
{:noreply, state}
101+
end
102+
end
103+
104+
def handle_cast(:record_success, state) do
105+
{_, current_state, _, _, _} = lookup!(state.skill_name)
106+
107+
case current_state do
108+
:half_open ->
109+
transition(state, :closed, 0, nil)
110+
111+
_ ->
112+
:ets.insert(@ets_table, {state.skill_name, current_state, 0, nil, DateTime.utc_now()})
113+
{:noreply, state}
114+
end
115+
end
116+
117+
@impl true
118+
def handle_call(:reset, _from, state) do
119+
cancel_timer(state.timer_ref)
120+
:ets.insert(@ets_table, {state.skill_name, :closed, 0, nil, DateTime.utc_now()})
121+
Logger.info("[CircuitBreaker] #{state.skill_name} manually reset to :closed")
122+
{:reply, :ok, %{state | timer_ref: nil}}
123+
end
124+
125+
@impl true
126+
def handle_info(:half_open, state) do
127+
:ets.insert(@ets_table, {state.skill_name, :half_open, 0, nil, DateTime.utc_now()})
128+
Logger.info("[CircuitBreaker] #{state.skill_name} → :half_open (testing)")
129+
{:noreply, %{state | timer_ref: nil}}
130+
end
131+
132+
# --- Internal ---
133+
134+
defp record_success(skill_name) do
135+
case Registry.lookup(AlexClaw.CircuitBreakerRegistry, skill_name) do
136+
[{pid, _}] -> GenServer.cast(pid, :record_success)
137+
[] -> :ok
138+
end
139+
end
140+
141+
defp record_failure(skill_name, reason) do
142+
case Registry.lookup(AlexClaw.CircuitBreakerRegistry, skill_name) do
143+
[{pid, _}] -> GenServer.cast(pid, {:record_failure, reason})
144+
[] -> :ok
145+
end
146+
end
147+
148+
defp transition(state, :open, count, reason) do
149+
cancel_timer(state.timer_ref)
150+
:ets.insert(@ets_table, {state.skill_name, :open, count, reason, DateTime.utc_now()})
151+
timer_ref = Process.send_after(self(), :half_open, @reset_timeout)
152+
153+
Logger.error(
154+
"[CircuitBreaker] #{state.skill_name} → :open after #{count} failures. " <>
155+
"Last error: #{inspect(reason)}. Auto-retry in #{div(@reset_timeout, 60_000)} min."
156+
)
157+
158+
notify_opened(state.skill_name, count, reason)
159+
{:noreply, %{state | timer_ref: timer_ref}}
160+
end
161+
162+
defp transition(state, :closed, count, _reason) do
163+
cancel_timer(state.timer_ref)
164+
:ets.insert(@ets_table, {state.skill_name, :closed, count, nil, DateTime.utc_now()})
165+
166+
Logger.info("[CircuitBreaker] #{state.skill_name} → :closed (recovered)")
167+
168+
notify_closed(state.skill_name)
169+
{:noreply, %{state | timer_ref: nil}}
170+
end
171+
172+
defp lookup!(skill_name) do
173+
case :ets.lookup(@ets_table, skill_name) do
174+
[entry] -> entry
175+
[] -> {skill_name, :closed, 0, nil, DateTime.utc_now()}
176+
end
177+
end
178+
179+
defp cancel_timer(nil), do: :ok
180+
defp cancel_timer(ref), do: Process.cancel_timer(ref)
181+
182+
defp via(skill_name) do
183+
{:via, Registry, {AlexClaw.CircuitBreakerRegistry, skill_name}}
184+
end
185+
186+
defp notify_opened(skill_name, count, reason) do
187+
Task.Supervisor.start_child(AlexClaw.TaskSupervisor, fn ->
188+
AlexClaw.Gateway.send_message(
189+
"⚡ Circuit *OPEN* for skill `#{skill_name}`\n" <>
190+
"#{count} consecutive failures. Auto-retry in #{div(@reset_timeout, 60_000)} min.\n" <>
191+
"Last error: `#{String.slice(inspect(reason), 0, 200)}`"
192+
)
193+
end)
194+
end
195+
196+
defp notify_closed(skill_name) do
197+
Task.Supervisor.start_child(AlexClaw.TaskSupervisor, fn ->
198+
AlexClaw.Gateway.send_message("✅ Circuit *CLOSED* for skill `#{skill_name}` — recovered.")
199+
end)
200+
end
201+
end

0 commit comments

Comments
 (0)