You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add real human-in-the-loop approval + UI/README polish
#4 Human-in-the-loop: when matching finds an exception, the run now PAUSES at
reconciliation (outcome "awaiting") instead of auto-posting. The dashboard shows
Approve / Reject and reconciliation only runs after a human clicks. Kept fully
stateless (no DB write): a two-phase request — phase 1 streams up to the pause,
the click fires phase 2 {id, decision} which re-runs the deterministic prefix and
continues into reconciliation, gated by a `humanApproval` input. reconcile() now
returns outcome posted | awaiting | rejected | blocked. Verified live (pause →
post on approve → un-posted on reject); stateless invariant still holds.
#1 Footer: reuse the GitHub/LinkedIn/email social icons from the invoice-parser
repo (components/social-links.tsx).
#5 Invoice queue: max-height + overflow-y-auto so it doesn't push the page tall.
#6 Fix the scenario label vanishing when a row is selected-but-not-yet-run.
#2 README: drop the PDF-parsing overclaim — intake validates/structures; parsing
lives in the sibling repo. #3 Add a Mermaid workflow diagram (incl. the human
gate). New README pattern section documenting the stateless HITL design.
Tests 75 (added lib/erp.test.ts for the gate + trace awaiting-status). All gates
green: typecheck · knip · test · sanity · build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Copy file name to clipboardExpand all lines: README.md
+36-19Lines changed: 36 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,6 @@
1
1
# ledgerloop
2
2
3
-
Four cooperating AI agents run an invoice through the **procure-to-pay** loop — intake → 2/3-way matching → approval routing → reconciliation — and the dashboard **streams the agent execution trace live** as it happens, including the moment a price/quantity mismatch is caught and the run is conditionally routed to human approval. Built with [Mastra](https://mastra.ai).
3
+
Four cooperating AI agents run an invoice through the **procure-to-pay** loop — intake → 2/3-way matching → approval routing → reconciliation — and the dashboard **streams the agent execution trace live** as it happens. When a price/quantity mismatch is caught, the run is conditionally routed to approval and **pauses for a real human decision** (Approve / Reject) before anything posts. Built with [Mastra](https://mastra.ai).
4
4
5
5
### ▶︎ [Try the live demo →](https://ledgerloop.vercel.app/)
6
6
@@ -14,33 +14,44 @@ Four cooperating AI agents run an invoice through the **procure-to-pay** loop
14
14
15
15
The demo simulates the **accounts-payable pipeline** an AP team runs on every vendor invoice:
1.**Intake**confirms and structures the invoice (the PDF-parsing role — the same job as the sibling [ai-invoice-parser](https://github.com/DylanMerigaud) repo).
39
+
1.**Intake**validates and structures the incoming invoice (vendor, line items, totals). *Parsing a PDF into this shape is the job of the sibling [ai-invoice-parser](https://github.com/DylanMerigaud) repo — this demo starts from the structured record and focuses on the orchestration.*
29
40
2.**Matching** runs a **2-way** (invoice ↔ PO) or **3-way** (invoice ↔ PO ↔ goods receipt) match and returns a verdict: `clean`, `exception`, or `duplicate`.
30
-
3.**Conditional routing** — *this is the demo*. A clean match goes **straight through** to reconciliation. An exception (a price variance, a quantity overbill, an off-PO line) is **routed to the Approval agent**, which tiers it to manager or director by the money and variance at stake. A duplicate is **blocked** so it's never paid twice.
31
-
4.**Reconciliation** posts the vendor bill + its double-entry GL distribution to the ERP (a fake adapter — see below) and returns the reference.
41
+
3.**Conditional routing — *this is the demo*.** A clean match goes **straight through** to reconciliation. An exception (a price variance, a quantity overbill, an off-PO line) is **routed to the Approval agent**, which tiers it to manager or director by the money and variance at stake — then the run **pauses for a human** to Approve or Reject before anything posts. A duplicate is **blocked** so it's never paid twice (no human needed — it's a control failure, not a judgment call).
42
+
4.**Reconciliation** posts the vendor bill + its double-entry GL distribution to the ERP (a fake adapter — see below) and returns the reference. It only runs once an invoice is cleared — auto (clean) or human-approved.
32
43
33
-
The split-view dashboard shows the **invoice queue** on the left (color-coded by outcome) and, on the right, the **live agent execution trace** for the selected invoice — each agent step, each tool call, the red "caught a mismatch" step, and the branch to approval — streamed in as the agents run.
44
+
The split-view dashboard shows the **invoice queue** on the left (color-coded by outcome) and, on the right, the **live agent execution trace** for the selected invoice — each agent step, each tool call, the red "caught a mismatch" step, the branch to approval, and the **pause where you click Approve / Reject** — streamed in as the agents run.
34
45
35
46
### The seeded scenario
36
47
37
48
The database is seeded with ~10 realistic invoices, **including three deliberate edge cases** — these are the demo:
38
49
39
50
| Invoice | Scenario | What the agents do |
40
51
| --- | --- | --- |
41
-
|`INV-2042`|**Price mismatch** — steel bar invoiced ~9% over the PO price | Matching flags `price_variance` → routed to **approval**|
42
-
|`INV-2048`|**Quantity mismatch** — invoiced 100 units, only 80 received | The **3-way** receipt check flags it → **director** approval |
43
-
|`INV-2041` (re-send) |**Duplicate** — the same invoice number arrives twice | Matching returns `duplicate` → **blocked**, not posted |
52
+
|`INV-2042`|**Price mismatch** — steel bar invoiced ~9% over the PO price | Matching flags `price_variance` → **manager** approval → **pauses for your decision**|
53
+
|`INV-2048`|**Quantity mismatch** — invoiced 100 units, only 80 received | The **3-way** receipt check flags it → **director** approval → **pauses for your decision**|
54
+
|`INV-2041` (re-send) |**Duplicate** — the same invoice number arrives twice | Matching returns `duplicate` → **blocked**, not posted (no human needed) |
@@ -82,7 +93,13 @@ The matching/approval/reconciliation **decisions** are **pure, unit-tested funct
82
93
83
94
The deliberate twist that keeps this **reliable enough to demo live**: the step also computes that same pure function directly and treats *that* as the authoritative result for routing ([`workflows/run-agent-step.ts`](src/mastra/workflows/run-agent-step.ts)). So the agent genuinely invokes its tool and writes the prose, but the verdict that drives the `.branch(...)` never depends on parsing a model response. If the model is slow, declines to call the tool, or the call fails entirely (no key, rate limit), the deterministic result and a fallback narration still stand — a flaky model degrades the prose, it never changes the routing or breaks the pipeline. Agents narrate and invoke; rules decide.
84
95
85
-
### 4. Zod as the single source of truth
96
+
### 4. A real human-in-the-loop — without persisting state
97
+
98
+
When matching finds an **exception**, the workflow runs intake → matching → approval and then **pauses at reconciliation** in an `awaiting` state — it does *not* post. The dashboard shows **Approve / Reject**, and reconciliation only executes after a human clicks (verified live: the ERP post genuinely doesn't happen until approval).
99
+
100
+
The interesting constraint: the demo is **stateless** (it never writes to the database, so every visitor sees pristine data — see below), yet a human pause normally implies persisting the suspended run somewhere to resume it later. ledgerloop resolves this with a **two-phase request** rather than a stored snapshot ([`api/run/route.ts`](app/api/run/route.ts)): phase 1 (`{ id }`) streams up to the pause; the Approve/Reject click fires phase 2 (`{ id, decision }`), which re-runs the cheap **deterministic** prefix (matching/approval are pure functions over the seeded bundle) and continues into reconciliation. The reviewer's decision flows through the workflow as a `humanApproval` input that gates the ERP post. Mastra also offers native `suspend`/`resume`, but that requires durable storage to span two serverless requests — recomputing a pure prefix is the stateless-friendly choice here, and a deliberate trade-off worth calling out.
101
+
102
+
### 5. Zod as the single source of truth
86
103
87
104
Every pipeline shape is defined once in Zod ([`lib/schema.ts`](lib/schema.ts)) and:
88
105
@@ -92,7 +109,7 @@ Every pipeline shape is defined once in Zod ([`lib/schema.ts`](lib/schema.ts)) a
92
109
93
110
The model, the validator, the database, and the screen can't drift — they're one definition. (This is the signature pattern carried over from the sibling repo, transposed from one extraction step to a four-stage pipeline.)
94
111
95
-
### 5. Native streaming, relayed and adapted
112
+
### 6. Native streaming, relayed and adapted
96
113
97
114
Mastra emits a native event stream for a workflow run. The [run route](app/api/run/route.ts) calls `run.stream()` and relays `result.fullStream` (a `ReadableStream` of typed chunks) to the browser as newline-delimited JSON. A small **adapter** ([`lib/trace.ts`](lib/trace.ts)) maps each raw Mastra chunk to a stable, UI-facing `TraceEvent` — so the dashboard depends on *our* vocabulary, not Mastra's internal chunk format, and an unrecognized chunk is dropped rather than crashing the stream. The client reads it with `fetch` + `response.body.getReader()`.
0 commit comments