Skip to content
Merged

v4 #97

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
ef6fb4f
Merge pull request #74 from propublica/main
aaronbrezel Mar 27, 2026
bdbe6b0
refactor: replace named-field PrepRecipeParams/RecipeParams with agno…
aaronbrezel Mar 27, 2026
d2d331b
feat: add cancelling to LoadingStatus
aaronbrezel Mar 31, 2026
fae58e6
feat: add cancel, isCancelled, setProgress to JobStore
aaronbrezel Mar 31, 2026
f689cd9
feat: add computeChunks helper with tests
aaronbrezel Mar 31, 2026
6905577
feat: pre-flight warning and chunked dispatch in handleRun
aaronbrezel Mar 31, 2026
e71a27e
refactor: inline runChunks IIFE and clarify chunk warning guard
aaronbrezel Mar 31, 2026
7154498
feat: stop button and cancelling state in JobIndicator
aaronbrezel Mar 31, 2026
ba76329
fix: escape double quotes in JobIndicator HTML attribute
aaronbrezel Mar 31, 2026
0a12d15
docs: add chunked batch execution design and implementation plan
aaronbrezel Mar 31, 2026
4745f65
feat: chunked batch execution with pre-flight warning and stop button…
aaronbrezel Apr 2, 2026
98d3c57
chore: merge origin/develop (PR #77) into local develop
aaronbrezel Apr 2, 2026
21c0b9c
fix: post-review cleanup for chunked batch execution (#77) (#78)
aaronbrezel Apr 2, 2026
3f1a370
Merge branch 'develop' of github.com:propublica/gas-ssi-toolkit into …
aaronbrezel Apr 2, 2026
e60fcf0
feat: add TokenInput searchable chip field for column selection (#52)
aaronbrezel Apr 2, 2026
100ed0e
docs: add PromptColList UI design for ordered prompt column selection
aaronbrezel Apr 2, 2026
cb36462
feat: replace split column pickers with ordered PromptColList in Conf…
aaronbrezel Apr 6, 2026
d1a9671
Merge branch 'develop' of github.com:propublica/gas-ssi-toolkit into …
aaronbrezel Apr 6, 2026
393e463
feat: reduce OAuth scopes to least-privilege for Drive access (#81)
aaronbrezel Apr 6, 2026
7b5a960
Merge branch 'develop' of github.com:propublica/gas-ssi-toolkit into …
aaronbrezel Apr 6, 2026
212e684
chore(deps-dev): bump hono from 4.12.8 to 4.12.12 (#82)
dependabot[bot] Apr 11, 2026
478a6f3
feat: Run AI Inference panel UX redesign (#85)
aaronbrezel Apr 11, 2026
3b43ec2
feat: column label prefix for user prompt parts (#86)
aaronbrezel Apr 14, 2026
63212b2
chore(deps-dev): bump @hono/node-server from 1.19.11 to 1.19.13 (#83)
dependabot[bot] Apr 14, 2026
b9bc1e1
chore(deps): bump brace-expansion (#73)
dependabot[bot] Apr 14, 2026
88e2fdf
feat: recipe architecture redesign — UserInput + prepTemplate + runTe…
aaronbrezel Apr 15, 2026
ee73ee2
feat: document summarization recipe redesign for journalism use (#88)
aaronbrezel Apr 15, 2026
3fe37c2
docs: update architecture.md and CLAUDE.md to reflect recipe redesign…
aaronbrezel Apr 16, 2026
8cab5ad
feat: improve empty-state message in Run AI panel with recipes link (…
aaronbrezel Apr 16, 2026
a0fce9b
hotfix - removing silent truncation of AI responses
aaronbrezel Apr 21, 2026
2f04ae7
Standardize font sizes, colors, and families (#92)
jfrankl Apr 22, 2026
d3e820e
docs: add CSS conventions section to CLAUDE.md (#93)
aaronbrezel Apr 22, 2026
cc9f0d1
feat: AI processing Phase 1 — parallel inference pipeline (#94)
aaronbrezel May 6, 2026
4e58aea
docs: recipe UX variants design spec
aaronbrezel May 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ Thumbs.db
# Gemini CLI internal
.gemini/

# Jest cache (redirect from system temp for sandbox compatibility)
.jest-cache/

# Git worktrees
.worktrees/

Expand Down
55 changes: 53 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,13 @@ src/client/sidebar-entry.ts (thin init — instantiates all panels, creates Rou
├── lockable-field.ts (LockableField — value + lock/unlock toggle; optional onUnlock callback)
├── recipe-prep-cook.ts (RecipePrepCook — 4-state machine: idle/prepping/prep-complete/cooking)
├── panel-loader.ts (PanelLoader — drives panel loading skeleton: progress bar, spinner, message)
└── job-indicator.ts (JobIndicator — renders active/failed jobs to #job-strip; persists across navigation)
├── job-indicator.ts (JobIndicator — renders active/failed jobs to #job-strip; persists across navigation)
├── token-input.ts (TokenInput — searchable chip field for column selection; multi or single-select;
│ supports includeNew for new column creation; use when item count is 8+ or items are
│ dynamic headers. Prefer TagList when count is small and all options benefit from
│ simultaneous display, e.g. the Tools section with ~5 fixed entries.)
└── prompt-col-list.ts (PromptColList — ordered list of PromptColumnSpec rows; each row pairs a
TokenInput column picker with a text/file kind toggle and reorder controls)
└── src/shared/types.ts

src/client/google.d.ts (compile-time type stub for google.script.run — uses declare global{} pattern)
Expand All @@ -155,12 +161,35 @@ The Gemini tool system spans three layers. `ToolId` (a string union in `shared/t

`buildGeminiPayload` in `api.ts` resolves `ToolId[]` via `TOOL_REGISTRY`, splits by `kind`, and assembles both shapes into the `tools` array of the REST request.

**Propagation path:** `ConfigureAIRunPanel` (UI TagList) → `RunConfig.tools` → `runBatchAI` → `runInference(tools?)` → `invokeGemini` → `callGeminiAPI` → `buildGeminiPayload`. For recipes: `RecipePanel` → `PrepRecipeParams.tools` → server echoes back as `PrepRecipeResult.tools` → assembled into `preppedRunConfig`.
**Propagation path:** `ConfigureAIRunPanel` (UI TagList) → `RunConfig.tools` → `runBatchAI` → `runInference(tools?)` → `invokeGemini` → `callGeminiAPI` → `buildGeminiPayload`. For recipes: `RecipePanel` → `PrepRecipeParams.cols + inputValues` → server resolves `inputId` references and writes columns → `PrepRecipeResult.rowRange` → client calls `buildRunTemplate(prepTemplate)` (derives `promptCols`/`systemPromptCol`/`outputCol` from `RecipeColumn.role`) merged with `definition.settings` and `rowRange` → `preppedRunConfig`.

Source files use relative imports (e.g. `../shared/types`). The `@server/*` and `@shared/*` aliases are **Jest-only** (mapped in `jest.config.cjs`) and are not available in TypeScript source.

Only `index.ts` should reference Google Apps Script UI services (SpreadsheetApp, HtmlService, PropertiesService). On the client side, only `services.ts` calls `google.script.run` (wrapping each call as a Promise); `sidebar-entry.ts` is a thin init file that creates the Router and calls `router.start()`.

### Recipe System

Recipes are journalist-facing presets that automate column setup and launch a Run AI. Each `RecipeDefinition` in `src/client/recipes.ts` has four parts:

| Field | Type | Purpose |
|-------|------|---------|
| `inputs` | `RecipeInput[]` | Journalist-facing form fields rendered by `RecipePanel` |
| `prepTemplate` | `RecipeColumn[]` | Column definitions sent to `prepRecipe()` on the server |
| `settings` | `RecipeSettings?` | Non-column `RunConfig` fields (tools, markdown, grounding, etc.) |
| Discovery fields | `id`, `name`, `icon`, `description`, `intro?` | Rendered in the recipes list and recipe header |

**`RecipeColumn`** (`src/client/types.ts`) = `PrepColSpec` + optional `role?: ColumnRole`. The `role` is client-only — `buildRunTemplate()` in `recipe.ts` maps roles to `promptCols`, `systemPromptCol`, and `outputCol` at cook time. Roles: `"file-prompt"` | `"text-prompt"` | `"system-prompt"` | `"output"`.

**`FillStrategy`** (`src/shared/types.ts`) controls how `prepRecipe()` populates each column:
- `{ kind: "fill-value"; value: string }` — writes a static string to every row
- `{ kind: "list-drive-folder"; inputId: string }` — lists files from the folder URL in `inputValues[inputId]`
- `{ kind: "create-empty" }` — creates the column with no content
- `{ kind: "template"; template: string }` — interpolates `{{inputId}}` placeholders; supports Mustache-style conditionals `{{#inputId}}...{{/inputId}}` (block is omitted when the input is empty)

**`RecipeInput.id`** must be camelCase or underscore_separated — no hyphens. The template interpolation regex uses `\w+` which does not match `-`.

**To add a new recipe:** add an entry to `RECIPES` in `src/client/recipes.ts`. No other files need changing unless you need a new `FillStrategy` kind.

### TypeScript Configuration

Two tsconfigs for two build environments:
Expand Down Expand Up @@ -256,3 +285,25 @@ Follows Google TypeScript Style Guide (enforced by ESLint + Prettier + pre-commi
- Semicolons required, double quotes (Prettier), trailing commas
- Explicit return types on functions (ESLint warning)
- Prefix unused parameters with `_`

## CSS Conventions

All sidebar styles live in `src/client/sidebar.css`. Use CSS custom properties — don't hardcode values.

**Font sizes** — pick the closest token, don't use raw px:

| Token | Value | Typical use |
|---|---|---|
| `--font-size-100` | 11px | Labels, badges, small metadata |
| `--font-size-200` | 12px | Helper text, tag chips, intro copy |
| `--font-size-300` | 14px | Body text, inputs, buttons (default) |
| `--font-size-400` | 16px | Panel titles |
| `--font-size-500` | 18px | Icons |

**Font family** — browsers don't inherit `font-family` on form elements. Any new `input`, `button`, `textarea`, or `select` must explicitly set `font-family: var(--font-family)`.

**Colors** — prefer variables over hardcoded hex:
- `--text-main` — primary text
- `--text-secondary` — secondary/muted text
- `--primary-blue` — interactive blue (`#1a73e8`)
- `--border-color` — borders and dividers
153 changes: 142 additions & 11 deletions __tests__/api.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

(globalThis as any).UrlFetchApp = {
fetch: jest.fn(),
fetchAll: jest.fn(),
};

(globalThis as any).PropertiesService = {
Expand All @@ -19,7 +20,12 @@

// ── Import after mocks ─────────────────────────────────────────

import { buildGeminiPayload, callGeminiAPI, invokeGemini } from "../src/server/api";
import {
buildGeminiPayload,
callGeminiAPI,
callGeminiAPIBatch,
invokeGemini,
} from "../src/server/api";
import { CONFIG } from "../src/server/config";
import type { GeminiRequest } from "../src/server/types";

Expand Down Expand Up @@ -186,19 +192,12 @@ describe("buildGeminiPayload", () => {
expect((payload.generationConfig as any).temperature).toBe(0.5);
});

it("applies CONFIG.MAX_OUTPUT_TOKENS as default maxOutputTokens when no generationConfig is provided", () => {
it("does not set maxOutputTokens when no generationConfig is provided", () => {
const payload = buildGeminiPayload(baseReq);
expect((payload.generationConfig as any).maxOutputTokens).toBe(CONFIG.MAX_OUTPUT_TOKENS);
expect((payload.generationConfig as any).maxOutputTokens).toBeUndefined();
});

it("applies CONFIG.MAX_OUTPUT_TOKENS as default when generationConfig omits maxOutputTokens", () => {
const req: GeminiRequest = { ...baseReq, generationConfig: { temperature: 0.7 } };
const payload = buildGeminiPayload(req);
expect((payload.generationConfig as any).maxOutputTokens).toBe(CONFIG.MAX_OUTPUT_TOKENS);
expect((payload.generationConfig as any).temperature).toBe(0.7);
});

it("uses caller-supplied maxOutputTokens over CONFIG default", () => {
it("passes through caller-supplied maxOutputTokens", () => {
const req: GeminiRequest = { ...baseReq, generationConfig: { maxOutputTokens: 512 } };
const payload = buildGeminiPayload(req);
expect((payload.generationConfig as any).maxOutputTokens).toBe(512);
Expand Down Expand Up @@ -362,3 +361,135 @@ describe("invokeGemini", () => {
expect(payload.contents[0].parts[1].inline_data.mime_type).toBe("application/pdf");
});
});

// ── callGeminiAPIBatch tests ───────────────────────────────────

function mockFetchAllResponses(bodies: unknown[]) {
(UrlFetchApp.fetchAll as jest.Mock).mockReturnValue(
bodies.map((body) => ({ getContentText: () => JSON.stringify(body) })),
);
}

describe("callGeminiAPIBatch", () => {
beforeEach(() => jest.clearAllMocks());

it("returns one GeminiResponse per request", () => {
mockFetchAllResponses([
{ candidates: [{ content: { parts: [{ text: "Result A" }] } }] },
{ candidates: [{ content: { parts: [{ text: "Result B" }] } }] },
]);
const reqs: GeminiRequest[] = [
{ apiKey: "key", userParts: [{ text: "Q1" }] },
{ apiKey: "key", userParts: [{ text: "Q2" }] },
];
const results = callGeminiAPIBatch(reqs);
expect(results).toHaveLength(2);
expect(results[0].text).toBe("Result A");
expect(results[1].text).toBe("Result B");
});

it("returns empty array for empty input", () => {
expect(callGeminiAPIBatch([])).toEqual([]);
expect(UrlFetchApp.fetchAll as jest.Mock).not.toHaveBeenCalled();
});

it("maps a Gemini error response to an error text result (does not throw)", () => {
mockFetchAllResponses([
{ error: { message: "quota exceeded" } },
{ candidates: [{ content: { parts: [{ text: "OK" }] } }] },
]);
const reqs: GeminiRequest[] = [
{ apiKey: "key", userParts: [{ text: "Q1" }] },
{ apiKey: "key", userParts: [{ text: "Q2" }] },
];
const results = callGeminiAPIBatch(reqs);
expect(results[0].text).toMatch(/Error:/);
expect(results[1].text).toBe("OK");
});

it("maps a non-JSON response to an error text result without aborting the batch", () => {
(UrlFetchApp.fetchAll as jest.Mock).mockReturnValue([
{
getResponseCode: () => 503,
getContentText: () => "<html>Service Unavailable</html>",
},
{
getContentText: () =>
JSON.stringify({ candidates: [{ content: { parts: [{ text: "OK" }] } }] }),
},
]);
const reqs: GeminiRequest[] = [
{ apiKey: "key", userParts: [{ text: "Q1" }] },
{ apiKey: "key", userParts: [{ text: "Q2" }] },
];
const results = callGeminiAPIBatch(reqs);
expect(results[0].text).toMatch(/Error:.*503/);
expect(results[1].text).toBe("OK");
});

it("includes file_data parts in the request payload", () => {
mockFetchAllResponses([{ candidates: [{ content: { parts: [{ text: "ok" }] } }] }]);
const req: GeminiRequest = {
apiKey: "key",
userParts: [
{ text: "Describe this file" },
{
file_data: {
file_uri: "https://generativelanguage.googleapis.com/v1beta/files/abc",
mime_type: "application/pdf",
},
},
],
};
callGeminiAPIBatch([req]);
const calls = (UrlFetchApp.fetchAll as jest.Mock).mock.calls[0][0];
const payload = JSON.parse(calls[0].payload);
expect(payload.contents[0].parts[1].file_data).toEqual({
file_uri: "https://generativelanguage.googleapis.com/v1beta/files/abc",
mime_type: "application/pdf",
});
});

it("uses modelName from request when provided", () => {
mockFetchAllResponses([{ candidates: [{ content: { parts: [{ text: "ok" }] } }] }]);
callGeminiAPIBatch([
{ apiKey: "key", modelName: "gemini-1.5-pro", userParts: [{ text: "Q" }] },
]);
const calls = (UrlFetchApp.fetchAll as jest.Mock).mock.calls[0][0];
expect(calls[0].url).toContain("gemini-1.5-pro");
});

it("falls back to CONFIG.MODEL_NAME when modelName is omitted", () => {
mockFetchAllResponses([{ candidates: [{ content: { parts: [{ text: "ok" }] } }] }]);
callGeminiAPIBatch([{ apiKey: "key", userParts: [{ text: "Q" }] }]);
const calls = (UrlFetchApp.fetchAll as jest.Mock).mock.calls[0][0];
expect(calls[0].url).toContain(CONFIG.MODEL_NAME);
});

it("returns 'No response.' when candidates are empty", () => {
mockFetchAllResponses([{ candidates: [] }]);
const results = callGeminiAPIBatch([{ apiKey: "key", userParts: [{ text: "Q" }] }]);
expect(results[0].text).toBe("No response.");
});

it("populates codePairs when executableCode and codeExecutionResult parts are present", () => {
mockFetchAllResponses([
{
candidates: [
{
content: {
parts: [
{ executableCode: { language: "PYTHON", code: "print(42)" } },
{ codeExecutionResult: { outcome: "OUTCOME_OK", output: "42\n" } },
],
},
},
],
},
]);
const results = callGeminiAPIBatch([{ apiKey: "key", userParts: [{ text: "Q" }] }]);
expect(results[0].codePairs).toHaveLength(1);
expect(results[0].codePairs![0].code.code).toBe("print(42)");
expect(results[0].codePairs![0].result.output).toBe("42\n");
});
});
Loading
Loading