Skip to content

Commit 991b164

Browse files
committed
docs + ci: changesets, READMEs, backend contract spec, CI workflow (RUN-384)
- changesets: minor bumps for @runflow-io/sdk (assets.upload, pin helpers), @runflow-io/proxy (allowedPaths), @runflow-io/studio (shell props, headless mask controller) - READMEs: SDK upload + pin-editing sections and API surface; proxy path-contract table + allowedPaths guide with security note; studio "Customizing the shell" + headless mask example; root README links the real-estate-studio-sdk reference repo as the worked fork example and documents the live proof gate - docs/plans/run-384-sdk-gaps/backend-contract.md: companion spec for backend gaps 2+6 — resolve runflow:// refs at model dispatch (read path already does), acceptance criteria, suggested ticket text - .github/workflows/ci.yml: bun install/build/typecheck/test/lint on PRs and main (first CI for this repo; live proof stays manual)
1 parent b317982 commit 991b164

9 files changed

Lines changed: 263 additions & 13 deletions

File tree

.changeset/proxy-allowed-paths.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@runflow-io/proxy": minor
3+
---
4+
5+
Add `allowedPaths` — an extensible, strictly-matched route allow-list on top of the built-ins (dispatch, run polling, health). Defaults now include the asset-upload pair `rf.assets.upload` needs (`POST /v1/asset-uploads`, `POST /v1/asset-uploads/:id/confirmations`); customer rules are additive, support method arrays and `:param` segments, and reject traversal. Org-data reads (run listing, billing) remain strictly opt-in. The handler now also exposes `PUT`/`PATCH`/`DELETE` for framework route exports. `RateLimitResult`'s `void` member is now `undefined` (type-level only).

.changeset/sdk-assets-pin.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
"@runflow-io/sdk": minor
3+
---
4+
5+
Add `rf.assets.upload(file)` — the browser-safe presigned upload flow (create session → PUT to storage → confirm), returning a model-ready signed HTTPS `url` plus the stable `runflow://assets/{id}` `ref`. Fixes the most common external-fork failure: browser file uploads ending up as `data:` URIs that models reject with a 422.
6+
7+
Export `composePinPrompt`, `composeRegionPrompt`, `pinRegion`, and `PinPoint` — the pin→region prompt convention (3×3 grid baked into the edit prompt) that previously existed only as private copies inside the studio bundle.

.changeset/studio-props-mask.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
---
2+
"@runflow-io/studio": minor
3+
---
4+
5+
`<StudioShell>` accepts four optional customization props — `tools` (workflow catalogue), `source` (initial asset URL or sample list), `sentinel` (`{ enabled, taskDescription }`), and `copy` (brand/labels) — making vertical forks possible without rebuilding on `./headless`. Zero props renders exactly as before. `mount()` forwards them via the new `props` option.
6+
7+
`./headless` now exports `createMaskController` — the framework-free dual-canvas brush engine (stroke interpolation, coverage, full-resolution thresholded mask blob) the shell itself uses, so headless consumers get working mask creation for inpaint workflows without rebuilding it.

.github/workflows/ci.yml

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
8+
concurrency:
9+
group: ci-${{ github.ref }}
10+
cancel-in-progress: true
11+
12+
jobs:
13+
checks:
14+
name: build · typecheck · test · lint
15+
runs-on: ubuntu-latest
16+
timeout-minutes: 15
17+
steps:
18+
- uses: actions/checkout@v4
19+
20+
- uses: oven-sh/setup-bun@v2
21+
with:
22+
bun-version: 1.3.10
23+
24+
- name: Install
25+
run: bun install --frozen-lockfile
26+
27+
- name: Build
28+
run: bun run build
29+
30+
- name: Typecheck
31+
run: bun run typecheck
32+
33+
- name: Test
34+
run: bun run test
35+
36+
- name: Lint
37+
run: bun run lint
38+
39+
# The live e2e proof (examples/e2e-proof) needs RUNFLOW_API_KEY and
40+
# spends real credits, so it stays a local/manual gate — see
41+
# `bun run proof`.

README.md

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,20 @@ Add a changeset for every user-visible change:
6262
bun changeset
6363
```
6464

65+
## End-to-end proof
66+
67+
`bun run proof` exercises the real customer chain — browser SDK →
68+
`@runflow-io/proxy` → api.runflow.io — across every modality, including
69+
file upload via `rf.assets.upload` and the proxy allow-list. It needs
70+
`RUNFLOW_API_KEY` and spends real credits, so it's a local/manual gate
71+
(not CI). Results land in `.proof/`.
72+
73+
For a worked example of a vertical fork (customize `<StudioShell>` via
74+
its `tools` / `source` / `sentinel` / `copy` props, or build a custom UI
75+
on `./headless`), see the
76+
[real-estate-studio-sdk](https://github.com/runflow-io/real-estate-studio-sdk)
77+
reference repo.
78+
6579
## License
6680

6781
MIT
Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
# Backend contract: resolve `runflow://` asset refs at dispatch (RUN-384 gaps 2 + 6)
2+
3+
**Target repo:** `runflow-monorepo` (api.runflow.io). **Not** implemented in `runflow-js`
4+
this document is the companion spec for the backend ticket.
5+
6+
## Current behavior (verified in code)
7+
8+
- **Writes** (`POST /v1/models/{owner}/{slug}/runs`, `POST /v1/comfyui-workflows/.../runs`,
9+
batches, admin retries): inline `data:` URIs in `body.input`/`body.metadata` are
10+
auto-materialized to R2 and stored as `runflow://assets/{uuid}` refs. This is correct
11+
and should stay.
12+
- **Reads** (`GET /v1/runs`, `GET /v1/runs/{id}`, batch/canonical listings): asset-backed
13+
URLs are re-signed and `runflow://assets/{uuid}` refs are resolved to short-TTL signed
14+
HTTPS URLs. Also correct.
15+
- **The gap:** at **model dispatch**, the materialized `runflow://` ref is forwarded to the
16+
model worker **as-is**. Worker media validators accept only HTTP(S)/`data:` URIs —
17+
`google/nano-banana-pro/edit` (and most non-ComfyUI models) reject with
18+
`422: media URL must use HTTP(S) or data URI, got 'runflow'`. ComfyUI workflow file
19+
inputs already accept the refs; singleton model dispatch does not.
20+
21+
So the `data:`-materialization *convenience* currently breaks the exact requests it
22+
rewrites. (The original ticket framed this as "make materialization opt-in"; the better
23+
fix below removes the need for any flag.)
24+
25+
## Requested change
26+
27+
At the dispatch layer — after materialization, before the input reaches the model
28+
worker / provider transport — apply the **same resolution the read path already does**:
29+
30+
1. Walk `body.input` (and `metadata` where it feeds workers) for strings of scheme
31+
`runflow://assets/{uuid}`.
32+
2. Resolve each ref org-scoped (existing read-side resolver semantics: load asset, check
33+
`access_expires_at`, sign `r2_key`) into a signed HTTPS URL whose TTL comfortably
34+
covers worker pull + retries (suggest ≥ the worker's max queue+run window).
35+
3. Forward the signed HTTPS URL to the worker. Persist the **ref** (not the signed URL)
36+
on the run record, as today.
37+
4. Unknown/foreign-org/expired refs → 422 with a precise message
38+
(`asset not found or expired: runflow://assets/{uuid}`) — fail at dispatch, not in
39+
the worker.
40+
41+
### Acceptance criteria
42+
43+
- `POST /v1/models/google/nano-banana-pro/edit/runs` with `input.image_urls:
44+
["runflow://assets/{uuid}"]` succeeds end to end (no 422), for both an explicit ref
45+
and one produced by `data:` auto-materialization.
46+
- Read-side responses are unchanged (refs still resolve on read).
47+
- ComfyUI dispatch behavior unchanged.
48+
- A run whose ref points at a foreign org's asset 404s/422s without leaking existence
49+
details beyond the standard non-leaky pattern.
50+
51+
### Why dispatch-side resolution (not validator changes, not an opt-in flag)
52+
53+
- One implementation point instead of N model-validator changes across providers.
54+
- Asset refs become first-class on the write path, matching the read path — the SDK can
55+
then hand `UploadedAsset.ref` (stable, no TTL) to any model instead of the signed `url`.
56+
- The `data:` materialization default stays a pure convenience with no footgun, so no
57+
config flag is needed.
58+
59+
## Interim state (already shipped in runflow-js)
60+
61+
`rf.assets.upload(file)` returns the **signed HTTPS** url from the confirmation response
62+
(`routers/asset_uploads.py` signs it server-side), so external forks are unblocked today
63+
without this change. Once dispatch-side resolution lands, the SDK will document `ref` as
64+
the preferred long-lived input.
65+
66+
## Suggested ticket
67+
68+
> **Title:** Resolve `runflow://assets/{uuid}` refs to signed HTTPS at model dispatch
69+
> **Parent:** RUN-384
70+
> **Why:** Auto-materialized `data:` inputs currently 422 on most singleton models
71+
> (worker media validators only accept HTTP(S)/data:). Read path already resolves refs;
72+
> dispatch must do the same so asset refs are first-class across the stack.
73+
> **Scope:** dispatch layer for `POST /v1/models/.../runs` (+ batches, retries);
74+
> resolver reuse from the read path; 422 on unknown/expired refs; tests per acceptance
75+
> criteria above. ComfyUI unchanged.

packages/proxy/README.md

Lines changed: 34 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -95,16 +95,43 @@ runflowProxy({
9595

9696
## Path contract
9797

98-
The proxy accepts only these paths (configurable via `allowedModels`):
98+
The proxy accepts these paths out of the box:
9999

100-
| Method | Path | Purpose |
101-
|--------|-----------------------------------|------------------|
102-
| POST | `/v1/models/{owner}/{slug…}/runs` | Dispatch a run. |
103-
| GET | `/v1/runs/{uuid}` | Poll a run. |
104-
| GET | `/v1/health` | Public health. |
100+
| Method | Path | Purpose |
101+
|--------|-----------------------------------------|----------------------------------|
102+
| POST | `/v1/models/{owner}/{slug…}/runs` | Dispatch a run. |
103+
| GET | `/v1/runs/{uuid}` | Poll a run. |
104+
| GET | `/v1/health` | Public health. |
105+
| POST | `/v1/asset-uploads` | Create a presigned upload. |
106+
| POST | `/v1/asset-uploads/{id}/confirmations` | Confirm it (`rf.assets.upload`). |
105107

106108
Everything else returns `403 Not allowed`. Run IDs are validated as
107-
UUIDv4-shape to block path traversal.
109+
UUIDv4-shape to block path traversal. Dispatch is additionally gated by
110+
`allowedModels`.
111+
112+
### Extending the allow-list: `allowedPaths`
113+
114+
```ts
115+
runflowProxy({
116+
apiKey: process.env.RUNFLOW_API_KEY!,
117+
allowedPaths: [
118+
{ method: "GET", path: "/v1/runs" }, // run listing
119+
{ method: "GET", path: "/v1/billing/balance" }, // billing read
120+
],
121+
});
122+
```
123+
124+
Rules are additive over the defaults and matched strictly — full path,
125+
segment by segment, no prefixes or wildcards. A `:param` segment matches
126+
exactly one non-empty segment and rejects traversal (`.`, `..`,
127+
percent-encoded forms). `method` takes a string or an array
128+
(`["GET", "DELETE"]`); the handler also exports `PUT`/`PATCH`/`DELETE`
129+
for framework route files.
130+
131+
> **Security:** every matched request is forwarded with **your** API
132+
> key, so an allowed `GET /v1/runs` exposes org-wide run data to any
133+
> same-origin browser session. Opt into reads deliberately and pair
134+
> them with `authenticate` + `rateLimit` in production.
108135
109136
## License
110137

packages/sdk/README.md

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,42 @@ const rf = new Runflow({ baseUrl: "/api/runflow" });
3434
The browser SDK never sees your API key — it's injected by
3535
`@runflow-io/proxy` on your server.
3636

37+
### Uploading files
38+
39+
```ts
40+
const asset = await rf.assets.upload(fileInput.files[0]);
41+
await rf.models.run("google/nano-banana-pro/edit", {
42+
input: { prompt: "remove the price tag", image_urls: [asset.url] },
43+
});
44+
```
45+
46+
`upload()` runs the platform's presigned flow (create session → PUT the
47+
bytes to storage → confirm) and returns `{ id, url, ref, ... }`. `url`
48+
is a short-TTL **signed HTTPS URL** — pass it straight to any model's
49+
media inputs. `ref` is the stable `runflow://assets/{id}` reference.
50+
Works in the browser through `@runflow-io/proxy` (the upload endpoints
51+
are on its default allow-list). Raw `Blob`s need `{ filename }`; the
52+
cap is 50 MB.
53+
54+
### Pin-based editing
55+
56+
Edit models like `google/nano-banana-pro/edit` have no `pin_x`/`pin_y`
57+
inputs — the convention is a region phrase baked into the prompt.
58+
`composePinPrompt` is that convention, shared with the Studio shell:
59+
60+
```ts
61+
import { composePinPrompt } from "@runflow-io/sdk";
62+
63+
await rf.models.run("google/nano-banana-pro/edit", {
64+
input: {
65+
// {x,y} are normalized 0..1; thirds map to upper|middle|lower ×
66+
// left|center|right ("upper-left" … "lower-right").
67+
prompt: composePinPrompt({ x: 0.25, y: 0.2 }, "remove the price tag"),
68+
image_urls: [sourceUrl],
69+
},
70+
});
71+
```
72+
3773
## Tools
3874

3975
Declarative model bindings with typed inputs, presets, and outputs:
@@ -109,8 +145,12 @@ information for `buildRequest` and the run helpers.
109145
- `runflow.models.run(model, body)` — dispatch a run. Model id segments
110146
are URL-encoded; `..`/empty segments are rejected.
111147
- `runflow.runs.get(id)` / `runflow.runs.poll(id)` / `runflow.runs.wait(id)`
148+
- `runflow.assets.upload(file, opts?)` — presigned upload; returns a
149+
signed https `url` + stable `runflow://` `ref`.
112150
- `runflow.tools.run(tool, args)` / `runflow.tools.dispatch(tool, args)`
113151
- `runflow.health.check()`
152+
- `pinRegion(pin)` / `composePinPrompt(pin, instruction)` — the shared
153+
pin→region prompt contract.
114154

115155
All return well-typed promises; errors are `RunflowError`,
116156
`RunFailedError`, or `RunTimeoutError`.

packages/studio/README.md

Lines changed: 40 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -66,12 +66,31 @@ mount(target: string | HTMLElement, options?: {
6666
theme?: ThemeMode | ThemeOverrides;
6767
/** Inject the default <style> into <head>. Default: true. */
6868
injectStyles?: boolean;
69+
/** Shell customization — see "Customizing the shell" below. */
70+
props?: StudioShellProps;
6971
}): { unmount(): void };
7072
```
7173

72-
That is the entire option surface today. If you need event callbacks,
73-
custom tool catalogues, the source picker hidden, or `update()` to
74-
reconfigure at runtime, those are tracked as follow-ups.
74+
## Customizing the shell
75+
76+
`<StudioShell>` (and `mount()`'s `props` option) takes four optional
77+
props; zero props renders the stock studio.
78+
79+
```tsx
80+
<StudioShell
81+
// The workflow catalogue: cards, chat-agent tools, package steps.
82+
tools={WORKFLOWS.filter((w) => w.group === "cleanup")}
83+
// A starting image URL, or SampleAsset[] to replace the samples.
84+
source="https://cdn.example/listing.jpg"
85+
// Disable quality evals, or re-template their task description.
86+
sentinel={{ enabled: false }}
87+
// Brand + labels, shallow-merged over the defaults.
88+
copy={{ brandName: "Estates Studio", brandTag: "", assetsTitle: "Listings" }}
89+
/>
90+
```
91+
92+
If you need event callbacks or `update()` to reconfigure at runtime,
93+
those are tracked as follow-ups.
7594

7695
## Companion endpoints
7796

@@ -150,9 +169,24 @@ const { output } = await rf.tools.run(tool, { image: "https://cdn/x.png" });
150169
```
151170

152171
The headless entry also exports `WORKFLOWS`, `SAMPLES`, the prototype's
153-
`runWorkflow` dispatcher, the `sentinelEvaluate` client, and
154-
`setStudioUrls`. A React state reducer for building a custom shell is
155-
tracked as a follow-up.
172+
`runWorkflow` dispatcher, the `sentinelEvaluate` client, `setStudioUrls`,
173+
and `createMaskController` — the framework-free dual-canvas brush engine
174+
the shell uses for its mask workflows:
175+
176+
```ts
177+
import { createMaskController } from "@runflow-io/studio/headless";
178+
179+
const mask = createMaskController({ brushSize: 45 });
180+
mask.attach(overlayCanvas); // rendered over your image
181+
mask.syncToDisplay(rect.width, rect.height, devicePixelRatio);
182+
// pointer events → beginStroke / strokeTo / endStroke; then:
183+
const blob = await mask.toMaskBlob(img.naturalWidth, img.naturalHeight);
184+
const asset = await rf.assets.upload(blob, { filename: "mask.png" });
185+
// → feed asset.url as mask_url to runflow/reference-inpaint etc.
186+
```
187+
188+
A React state reducer for building a custom shell is tracked as a
189+
follow-up.
156190

157191
## License
158192

0 commit comments

Comments
 (0)