Skip to content

Commit e328ac2

Browse files
DylanMerigaudclaude
andcommitted
Add real human-in-the-loop approval + UI/README polish
#4 Human-in-the-loop: when matching finds an exception, the run now PAUSES at reconciliation (outcome "awaiting") instead of auto-posting. The dashboard shows Approve / Reject and reconciliation only runs after a human clicks. Kept fully stateless (no DB write): a two-phase request — phase 1 streams up to the pause, the click fires phase 2 {id, decision} which re-runs the deterministic prefix and continues into reconciliation, gated by a `humanApproval` input. reconcile() now returns outcome posted | awaiting | rejected | blocked. Verified live (pause → post on approve → un-posted on reject); stateless invariant still holds. #1 Footer: reuse the GitHub/LinkedIn/email social icons from the invoice-parser repo (components/social-links.tsx). #5 Invoice queue: max-height + overflow-y-auto so it doesn't push the page tall. #6 Fix the scenario label vanishing when a row is selected-but-not-yet-run. #2 README: drop the PDF-parsing overclaim — intake validates/structures; parsing lives in the sibling repo. #3 Add a Mermaid workflow diagram (incl. the human gate). New README pattern section documenting the stateless HITL design. Tests 75 (added lib/erp.test.ts for the gate + trace awaiting-status). All gates green: typecheck · knip · test · sanity · build. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent b91dea1 commit e328ac2

18 files changed

Lines changed: 488 additions & 105 deletions

README.md

Lines changed: 36 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# ledgerloop
22

3-
Four cooperating AI agents run an invoice through the **procure-to-pay** loop — intake → 2/3-way matching → approval routing → reconciliation — and the dashboard **streams the agent execution trace live** as it happens, including the moment a price/quantity mismatch is caught and the run is conditionally routed to human approval. Built with [Mastra](https://mastra.ai).
3+
Four cooperating AI agents run an invoice through the **procure-to-pay** loop — intake → 2/3-way matching → approval routing → reconciliation — and the dashboard **streams the agent execution trace live** as it happens. When a price/quantity mismatch is caught, the run is conditionally routed to approval and **pauses for a real human decision** (Approve / Reject) before anything posts. Built with [Mastra](https://mastra.ai).
44

55
### ▶︎ [Try the live demo →](https://ledgerloop.vercel.app/)
66

@@ -14,33 +14,44 @@ Four cooperating AI agents run an invoice through the **procure-to-pay** loop
1414

1515
The demo simulates the **accounts-payable pipeline** an AP team runs on every vendor invoice:
1616

17-
```
18-
┌──────────┐ ┌───────────┐ ┌────────────┐ ┌─────────────────┐
19-
PDF → │ Intake │ → │ Matching │ → │ Approval │ → │ Reconciliation │ → ERP
20-
│ agent │ │ agent │ │ agent │ │ agent │
21-
└──────────┘ └─────┬─────┘ └────────────┘ └─────────────────┘
22-
│ ▲
23-
│ clean match │ price / qty / duplicate
24-
└──────────────────┴── exception → route to a human
25-
straight-through
17+
```mermaid
18+
flowchart LR
19+
I([Invoice]) --> A1[Intake agent]
20+
A1 --> A2[Matching agent]
21+
A2 -- "run-match tool" --> V{Verdict?}
22+
23+
V -- "clean" --> AUTO[Auto-approve<br/>straight-through]
24+
V -- "exception<br/>price / qty / off-PO" --> A3[Approval agent<br/>tier: manager / director]
25+
V -- "duplicate" --> BLK[Blocked<br/>never posted]
26+
27+
A3 --> H{{Human decision<br/>Approve / Reject}}
28+
H -- "approve" --> A4
29+
H -- "reject" --> REJ[Rejected<br/>not posted]
30+
AUTO --> A4[Reconciliation agent]
31+
A4 -- "post-to-erp tool" --> ERP([ERP: vendor bill + GL])
32+
33+
classDef gate fill:#FEF3C7,stroke:#B45309,color:#0A0A0A;
34+
classDef stop fill:#FEE2E2,stroke:#B91C1C,color:#0A0A0A;
35+
class H gate;
36+
class BLK,REJ stop;
2637
```
2738

28-
1. **Intake** confirms and structures the invoice (the PDF-parsing role — the same job as the sibling [ai-invoice-parser](https://github.com/DylanMerigaud) repo).
39+
1. **Intake** validates and structures the incoming invoice (vendor, line items, totals). *Parsing a PDF into this shape is the job of the sibling [ai-invoice-parser](https://github.com/DylanMerigaud) repo — this demo starts from the structured record and focuses on the orchestration.*
2940
2. **Matching** runs a **2-way** (invoice ↔ PO) or **3-way** (invoice ↔ PO ↔ goods receipt) match and returns a verdict: `clean`, `exception`, or `duplicate`.
30-
3. **Conditional routing***this is the demo*. A clean match goes **straight through** to reconciliation. An exception (a price variance, a quantity overbill, an off-PO line) is **routed to the Approval agent**, which tiers it to manager or director by the money and variance at stake. A duplicate is **blocked** so it's never paid twice.
31-
4. **Reconciliation** posts the vendor bill + its double-entry GL distribution to the ERP (a fake adapter — see below) and returns the reference.
41+
3. **Conditional routing — *this is the demo*.** A clean match goes **straight through** to reconciliation. An exception (a price variance, a quantity overbill, an off-PO line) is **routed to the Approval agent**, which tiers it to manager or director by the money and variance at stake — then the run **pauses for a human** to Approve or Reject before anything posts. A duplicate is **blocked** so it's never paid twice (no human needed — it's a control failure, not a judgment call).
42+
4. **Reconciliation** posts the vendor bill + its double-entry GL distribution to the ERP (a fake adapter — see below) and returns the reference. It only runs once an invoice is cleared — auto (clean) or human-approved.
3243

33-
The split-view dashboard shows the **invoice queue** on the left (color-coded by outcome) and, on the right, the **live agent execution trace** for the selected invoice — each agent step, each tool call, the red "caught a mismatch" step, and the branch to approval — streamed in as the agents run.
44+
The split-view dashboard shows the **invoice queue** on the left (color-coded by outcome) and, on the right, the **live agent execution trace** for the selected invoice — each agent step, each tool call, the red "caught a mismatch" step, the branch to approval, and the **pause where you click Approve / Reject** — streamed in as the agents run.
3445

3546
### The seeded scenario
3647

3748
The database is seeded with ~10 realistic invoices, **including three deliberate edge cases** — these are the demo:
3849

3950
| Invoice | Scenario | What the agents do |
4051
| --- | --- | --- |
41-
| `INV-2042` | **Price mismatch** — steel bar invoiced ~9% over the PO price | Matching flags `price_variance`routed to **approval** |
42-
| `INV-2048` | **Quantity mismatch** — invoiced 100 units, only 80 received | The **3-way** receipt check flags it → **director** approval |
43-
| `INV-2041` (re-send) | **Duplicate** — the same invoice number arrives twice | Matching returns `duplicate`**blocked**, not posted |
52+
| `INV-2042` | **Price mismatch** — steel bar invoiced ~9% over the PO price | Matching flags `price_variance`**manager** approval → **pauses for your decision** |
53+
| `INV-2048` | **Quantity mismatch** — invoiced 100 units, only 80 received | The **3-way** receipt check flags it → **director** approval **pauses for your decision** |
54+
| `INV-2041` (re-send) | **Duplicate** — the same invoice number arrives twice | Matching returns `duplicate`**blocked**, not posted (no human needed) |
4455
| 6 × clean | Clean 2-way / 3-way matches | Auto-approved → **straight-through** to reconciliation |
4556

4657
---
@@ -82,7 +93,13 @@ The matching/approval/reconciliation **decisions** are **pure, unit-tested funct
8293

8394
The deliberate twist that keeps this **reliable enough to demo live**: the step also computes that same pure function directly and treats *that* as the authoritative result for routing ([`workflows/run-agent-step.ts`](src/mastra/workflows/run-agent-step.ts)). So the agent genuinely invokes its tool and writes the prose, but the verdict that drives the `.branch(...)` never depends on parsing a model response. If the model is slow, declines to call the tool, or the call fails entirely (no key, rate limit), the deterministic result and a fallback narration still stand — a flaky model degrades the prose, it never changes the routing or breaks the pipeline. Agents narrate and invoke; rules decide.
8495

85-
### 4. Zod as the single source of truth
96+
### 4. A real human-in-the-loop — without persisting state
97+
98+
When matching finds an **exception**, the workflow runs intake → matching → approval and then **pauses at reconciliation** in an `awaiting` state — it does *not* post. The dashboard shows **Approve / Reject**, and reconciliation only executes after a human clicks (verified live: the ERP post genuinely doesn't happen until approval).
99+
100+
The interesting constraint: the demo is **stateless** (it never writes to the database, so every visitor sees pristine data — see below), yet a human pause normally implies persisting the suspended run somewhere to resume it later. ledgerloop resolves this with a **two-phase request** rather than a stored snapshot ([`api/run/route.ts`](app/api/run/route.ts)): phase 1 (`{ id }`) streams up to the pause; the Approve/Reject click fires phase 2 (`{ id, decision }`), which re-runs the cheap **deterministic** prefix (matching/approval are pure functions over the seeded bundle) and continues into reconciliation. The reviewer's decision flows through the workflow as a `humanApproval` input that gates the ERP post. Mastra also offers native `suspend`/`resume`, but that requires durable storage to span two serverless requests — recomputing a pure prefix is the stateless-friendly choice here, and a deliberate trade-off worth calling out.
101+
102+
### 5. Zod as the single source of truth
86103

87104
Every pipeline shape is defined once in Zod ([`lib/schema.ts`](lib/schema.ts)) and:
88105

@@ -92,7 +109,7 @@ Every pipeline shape is defined once in Zod ([`lib/schema.ts`](lib/schema.ts)) a
92109

93110
The model, the validator, the database, and the screen can't drift — they're one definition. (This is the signature pattern carried over from the sibling repo, transposed from one extraction step to a four-stage pipeline.)
94111

95-
### 5. Native streaming, relayed and adapted
112+
### 6. Native streaming, relayed and adapted
96113

97114
Mastra emits a native event stream for a workflow run. The [run route](app/api/run/route.ts) calls `run.stream()` and relays `result.fullStream` (a `ReadableStream` of typed chunks) to the browser as newline-delimited JSON. A small **adapter** ([`lib/trace.ts`](lib/trace.ts)) maps each raw Mastra chunk to a stable, UI-facing `TraceEvent` — so the dashboard depends on *our* vocabulary, not Mastra's internal chunk format, and an unrecognized chunk is dropped rather than crashing the stream. The client reads it with `fetch` + `response.body.getReader()`.
98115

app/api/run/route.ts

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -38,15 +38,23 @@ function line(obj: unknown): Uint8Array {
3838
export async function POST(request: Request): Promise<Response> {
3939
// 1. Parse + validate the request body.
4040
let id: string;
41+
let decision: "approve" | "reject" | undefined;
4142
try {
4243
const body: unknown = await request.json();
43-
id = RunRequest.parse(body).id;
44+
const parsed = RunRequest.parse(body);
45+
id = parsed.id;
46+
decision = parsed.decision;
4447
} catch {
4548
return Response.json(
46-
{ error: "Body must be { id: string } naming a seeded invoice." },
49+
{ error: "Body must be { id: string, decision?: \"approve\" | \"reject\" }." },
4750
{ status: 400 },
4851
);
4952
}
53+
// The reviewer's decision maps to the workflow's humanApproval input. No
54+
// decision → "pending" (an exception pauses for a human). "approve"/"reject"
55+
// are phase-2 resumes. Recomputing the cheap deterministic prefix instead of
56+
// restoring a persisted snapshot is what keeps the human-in-the-loop stateless.
57+
const humanApproval = decision ?? "pending";
5058

5159
// 2. Load the seeded document bundle (READ ONLY). Done before opening the
5260
// stream so a missing invoice / missing DB config is a clean HTTP error
@@ -88,6 +96,7 @@ export async function POST(request: Request): Promise<Response> {
8896
purchaseOrder: bundle.purchaseOrder,
8997
goodsReceipt: bundle.goodsReceipt,
9098
priorInvoiceNumbers: bundle.priorInvoiceNumbers,
99+
humanApproval,
91100
},
92101
});
93102

app/page.tsx

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
import { Dashboard } from "@/components/dashboard";
2+
import { SocialLinks } from "@/components/social-links";
23
import { listInvoiceQueue, type QueueItem } from "@/db/client";
34
import { PIPELINE_MODEL } from "@/src/mastra/model";
45

@@ -109,16 +110,10 @@ function Footer() {
109110
<span className="font-mono text-ink">{PIPELINE_MODEL}</span> · Next.js · Supabase ·
110111
Drizzle. Runs are stateless — nothing is written back.
111112
</p>
112-
<p>
113-
<a
114-
className="text-accent hover:underline"
115-
href="https://github.com/DylanMerigaud"
116-
target="_blank"
117-
rel="noreferrer"
118-
>
119-
Dylan Mérigaud
120-
</a>
121-
</p>
113+
<div className="flex items-center gap-2">
114+
<span className="text-ink">Dylan Mérigaud</span>
115+
<SocialLinks />
116+
</div>
122117
</footer>
123118
);
124119
}

components/dashboard.tsx

Lines changed: 42 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ import type { QueueItem } from "@/db/client";
2121

2222
export function Dashboard({ queue }: { queue: QueueItem[] }) {
2323
const [selectedId, setSelectedId] = useState<string | null>(queue[0]?.id ?? null);
24-
const { state, run, reset } = usePipelineRun();
24+
const { state, run, decide, reset } = usePipelineRun();
2525

2626
const selected = queue.find((q) => q.id === selectedId) ?? null;
2727

@@ -39,7 +39,7 @@ export function Dashboard({ queue }: { queue: QueueItem[] }) {
3939
<CardTitle>Invoice queue</CardTitle>
4040
<span className="text-[11px] text-muted tnum">{queue.length} invoices</span>
4141
</CardHeader>
42-
<ul className="divide-y divide-line">
42+
<ul className="max-h-[calc(100vh-16rem)] divide-y divide-line overflow-y-auto">
4343
{queue.map((item) => {
4444
const isSelected = item.id === selectedId;
4545
// The pill reflects the live run only for the selected row; others
@@ -73,11 +73,15 @@ export function Dashboard({ queue }: { queue: QueueItem[] }) {
7373
{item.invoiceNumber}
7474
{item.poNumber ? ` · ${item.poNumber}` : ""}
7575
</span>
76-
{item.scenario && !isSelected && (
77-
<span className="shrink-0 text-[10px] text-muted/80">{item.scenario}</span>
78-
)}
79-
{isSelected && state.status !== "idle" && (
76+
{/* Once a run is active for the selected row, show its live
77+
outcome badge; otherwise always show the seeded scenario
78+
hint (so selecting a row never blanks the label). */}
79+
{isSelected && state.status !== "idle" ? (
8080
<Badge tone={outcomeTone(outcome)}>{outcomeLabel(outcome)}</Badge>
81+
) : (
82+
item.scenario && (
83+
<span className="shrink-0 text-[10px] text-muted/80">{item.scenario}</span>
84+
)
8185
)}
8286
</span>
8387
</span>
@@ -99,18 +103,38 @@ export function Dashboard({ queue }: { queue: QueueItem[] }) {
99103
</p>
100104
)}
101105
</div>
102-
<button
103-
type="button"
104-
disabled={!selected || state.status === "running"}
105-
onClick={() => selected && run(selected.id)}
106-
className="shrink-0 rounded-lg bg-accent px-3.5 py-2 text-[13px] font-medium text-accent-fg shadow-sm transition-opacity hover:opacity-90 disabled:cursor-not-allowed disabled:opacity-40"
107-
>
108-
{state.status === "running"
109-
? "Running…"
110-
: state.status === "done" || state.status === "error"
111-
? "Run again"
112-
: "Run pipeline"}
113-
</button>
106+
{state.status === "awaiting" && selected ? (
107+
// The run paused for a human decision — show the approval gate.
108+
<div className="flex shrink-0 items-center gap-2">
109+
<button
110+
type="button"
111+
onClick={() => decide(selected.id, "reject")}
112+
className="rounded-lg px-3 py-2 text-[13px] font-medium text-danger ring-1 ring-inset ring-danger-line transition-colors hover:bg-danger-soft"
113+
>
114+
Reject
115+
</button>
116+
<button
117+
type="button"
118+
onClick={() => decide(selected.id, "approve")}
119+
className="rounded-lg bg-ok px-3.5 py-2 text-[13px] font-medium text-white shadow-sm transition-opacity hover:opacity-90"
120+
>
121+
Approve
122+
</button>
123+
</div>
124+
) : (
125+
<button
126+
type="button"
127+
disabled={!selected || state.status === "running"}
128+
onClick={() => selected && run(selected.id)}
129+
className="shrink-0 rounded-lg bg-accent px-3.5 py-2 text-[13px] font-medium text-accent-fg shadow-sm transition-opacity hover:opacity-90 disabled:cursor-not-allowed disabled:opacity-40"
130+
>
131+
{state.status === "running"
132+
? "Running…"
133+
: state.status === "done" || state.status === "error"
134+
? "Run again"
135+
: "Run pipeline"}
136+
</button>
137+
)}
114138
</CardHeader>
115139
<div className="flex-1 overflow-y-auto px-5 py-4">
116140
<TraceTimeline state={state} invoiceLabel={selected?.invoiceNumber ?? null} />

components/social-links.tsx

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
/**
2+
* Compact contact icons (GitHub, LinkedIn, email) for the footer — muted by
3+
* default, indigo on hover. Icons only, no labels, to keep it sober. Mirrors the
4+
* sibling ai-invoice-parser repo so the two demos share one footer language.
5+
*/
6+
7+
const CONTACT = {
8+
github: "https://github.com/DylanMerigaud",
9+
linkedin: "https://www.linkedin.com/in/dylanmerigaud/",
10+
email: "dylanmerigaud.pro@gmail.com",
11+
} as const;
12+
13+
export function SocialLinks() {
14+
return (
15+
<nav aria-label="Contact" className="flex items-center gap-1">
16+
<IconLink href={CONTACT.github} label="GitHub" external>
17+
<GitHubIcon />
18+
</IconLink>
19+
<IconLink href={CONTACT.linkedin} label="LinkedIn" external>
20+
<LinkedInIcon />
21+
</IconLink>
22+
<IconLink href={`mailto:${CONTACT.email}`} label="Email">
23+
<MailIcon />
24+
</IconLink>
25+
</nav>
26+
);
27+
}
28+
29+
function IconLink({
30+
href,
31+
label,
32+
external = false,
33+
children,
34+
}: {
35+
href: string;
36+
label: string;
37+
external?: boolean;
38+
children: React.ReactNode;
39+
}) {
40+
return (
41+
<a
42+
href={href}
43+
aria-label={label}
44+
title={label}
45+
{...(external ? { target: "_blank", rel: "noreferrer noopener" } : {})}
46+
className="flex h-8 w-8 items-center justify-center rounded-lg text-muted transition-colors hover:bg-accent-soft hover:text-accent"
47+
>
48+
{children}
49+
</a>
50+
);
51+
}
52+
53+
function GitHubIcon() {
54+
return (
55+
<svg width="17" height="17" viewBox="0 0 24 24" fill="currentColor" aria-hidden>
56+
<path d="M12 .5C5.37.5 0 5.87 0 12.5c0 5.3 3.44 9.8 8.21 11.39.6.11.82-.26.82-.58 0-.29-.01-1.04-.02-2.04-3.34.73-4.04-1.61-4.04-1.61-.55-1.39-1.33-1.76-1.33-1.76-1.09-.74.08-.73.08-.73 1.2.09 1.84 1.24 1.84 1.24 1.07 1.83 2.81 1.3 3.49.99.11-.78.42-1.3.76-1.6-2.67-.3-5.47-1.33-5.47-5.93 0-1.31.47-2.38 1.24-3.22-.12-.31-.54-1.52.12-3.18 0 0 1.01-.32 3.3 1.23a11.5 11.5 0 0 1 6.01 0c2.29-1.55 3.3-1.23 3.3-1.23.66 1.66.24 2.87.12 3.18.77.84 1.23 1.91 1.23 3.22 0 4.61-2.81 5.62-5.49 5.92.43.37.81 1.1.81 2.22 0 1.61-.01 2.9-.01 3.29 0 .32.21.7.82.58A12 12 0 0 0 24 12.5C24 5.87 18.63.5 12 .5z" />
57+
</svg>
58+
);
59+
}
60+
61+
function LinkedInIcon() {
62+
return (
63+
<svg width="17" height="17" viewBox="0 0 24 24" fill="currentColor" aria-hidden>
64+
<path d="M20.45 20.45h-3.56v-5.57c0-1.33-.02-3.04-1.85-3.04-1.85 0-2.14 1.45-2.14 2.94v5.67H9.35V9h3.42v1.56h.05c.48-.9 1.64-1.85 3.37-1.85 3.6 0 4.27 2.37 4.27 5.46v6.28zM5.34 7.43a2.06 2.06 0 1 1 0-4.13 2.06 2.06 0 0 1 0 4.13zM7.12 20.45H3.56V9h3.56v11.45zM22.22 0H1.77C.79 0 0 .77 0 1.73v20.54C0 23.23.79 24 1.77 24h20.45c.98 0 1.78-.77 1.78-1.73V1.73C24 .77 23.2 0 22.22 0z" />
65+
</svg>
66+
);
67+
}
68+
69+
function MailIcon() {
70+
return (
71+
<svg
72+
width="17"
73+
height="17"
74+
viewBox="0 0 24 24"
75+
fill="none"
76+
stroke="currentColor"
77+
strokeWidth="2"
78+
strokeLinecap="round"
79+
strokeLinejoin="round"
80+
aria-hidden
81+
>
82+
<rect x="2" y="4" width="20" height="16" rx="2" />
83+
<path d="m22 7-8.97 5.7a1.94 1.94 0 0 1-2.06 0L2 7" />
84+
</svg>
85+
);
86+
}

0 commit comments

Comments
 (0)