Merge pull request #72 from propublica/develop

V3

Author: aaronbrezel

.clasp.json

@@ -24,4 +24,7 @@ Thumbs.db
 .gemini/
 
 # Git worktrees
-.worktrees/
+.worktrees/
+
+# Clasp project file (contains project ID and rootDir)
+.clasp.json

.gitignore

@@ -82,14 +82,12 @@ HtmlService can only serve `.html` files — all JavaScript and CSS must be inli
 4. Emits `dist/Sidebar.html` as an asset
 5. Deletes the intermediate `.js` chunk so clasp never pushes it as a `.gs` file
 
-The `src/Sidebar.html` template is also read at test time by `__tests__/helpers/sidebar-fixtures.ts` to keep DOM fixtures structurally in sync with the real template.
-
 ### Module Dependency Graph
 
 **Server:**
 ```
 src/server/index.ts          (entry point — menu, 4 tool orchestrators, UI handlers, re-exports custom functions)
-├── src/server/config.ts         (CONFIG object: API key property name, model, column names, limits)
+├── src/server/config.ts         (CONFIG object: API key property name, model name, size limits)
 ├── src/server/api.ts            (callGeminiAPI, buildGeminiPayload, invokeGemini — pure HTTP adapter via UrlFetchApp;
 │                                 buildGeminiPayload resolves ToolId[] via TOOL_REGISTRY, splits grounding vs function tools)
 ├── src/server/inference.ts      (runInference — unified inference handler for menu-triggered AI calls; no SpreadsheetApp dep;
@@ -99,13 +97,14 @@ src/server/index.ts          (entry point — menu, 4 tool orchestrators, UI han
 ├── src/server/types.ts          (server-only types: AppConfig, GeminiRequest, GeminiTool discriminated union,
 │                                 GeminiInlineData, GeminiFunctionDeclaration, DriveFileInfo; never imported by client)
 ├── src/server/drive.ts          (extractTextUniversal, fetchAndEncodeFile, checkDriveService)
-├── src/server/dialog.ts         (HTML_TEMPLATE string for AI mode selection modal)
+├── src/server/rich-text.ts      (CellContent, TextRange interfaces; buildRichInferenceCellContent, buildRichGroundingCellContent —
+│                                 pure layer between GeminiResponse and Sheets cell content; no GAS globals)
 ├── src/server/customFunctions.ts  (SSI — Sheets custom function; calls invokeGemini directly; always returns string,
 │                                 uses "[SSI Error: ...]" format)
 ├── src/server/utils.ts          (extractId, isValidDriveLink, createSeededRandom, getAllFilesRecursive, sampleRows,
 │                                 truncateText, findOrCreateColumn, writeColumn, flattenArg)
-└── src/shared/types.ts          (RPC boundary ONLY — ToolId union, RunConfig, PrepRecipeParams, PrepRecipeResult;
-                                  all with optional tools?: ToolId[])
+└── src/shared/types.ts          (RPC boundary ONLY — ToolId union, RunConfig, PrepRecipeParams, PrepRecipeResult,
+                                  ImportDriveLinksConfig, ExtractTextConfig; all with optional tools?: ToolId[])
 ```
 
 **Client:**
@@ -118,17 +117,22 @@ src/client/sidebar-entry.ts  (thin init — instantiates all panels, creates Rou
 └── src/client/tools.ts          (TOOL_CATALOG: ToolCatalogEntry[] — display metadata for sidebar TagList;
 │                                 hardcoded at build time, no RPC needed)
 └── src/client/recipes.ts        (RECIPES registry — RecipeDefinition[] for all standard recipes)
+└── src/client/job-store.ts      (JobStore class — tracks active jobs, polls getJobProgress, notifies subscribers)
 └── src/client/panels/
 │   ├── tool-list.ts             (ToolListPanel — entry screen, dispatches to tool or recipes)
 │   ├── configure-ai-run.ts      (ConfigureAIRunPanel — column mapping, row range, tool selection, AI run)
+│   ├── import-drive-links.ts    (ImportDriveLinksPanel — Drive folder import UI; mime-type filter, output column)
+│   ├── extract-text.ts          (ExtractTextPanel — text extraction UI; source/output column, row range)
 │   ├── recipes-list.ts          (RecipesListPanel — browsable list of recipes)
 │   └── recipe.ts                (RecipePanel — generic panel driven by RecipeParams; prep → cook flow)
 └── src/client/components/       (reusable UI components)
     ├── tag-list.ts              (TagList — multi-select tag chips; accepts string[] or {label,value}[] items)
     ├── single-tag-list.ts       (SingleTagList — exclusive-select tag chips)
     ├── row-range.ts             (RowRange — start/end row inputs)
     ├── lockable-field.ts        (LockableField — value + lock/unlock toggle; optional onUnlock callback)
-    └── recipe-prep-cook.ts      (RecipePrepCook — 4-state machine: idle/prepping/prep-complete/cooking)
+    ├── recipe-prep-cook.ts      (RecipePrepCook — 4-state machine: idle/prepping/prep-complete/cooking)
+    ├── panel-loader.ts          (PanelLoader — drives panel loading skeleton: progress bar, spinner, message)
+    └── job-indicator.ts         (JobIndicator — renders active/failed jobs to #job-strip; persists across navigation)
     └── src/shared/types.ts
 
 src/client/google.d.ts       (compile-time type stub for google.script.run — uses declare global{} pattern)
@@ -194,13 +198,6 @@ beforeEach(() => {
 // Then invoke capturedSuccess(...) / capturedFailure(...) to simulate GAS callbacks.
 ```
 
-**Shared DOM fixtures:** `__tests__/helpers/sidebar-fixtures.ts` exports:
-- `FULL_SIDEBAR_HTML` — the sidebar HTML template read from `src/Sidebar.html` at test time (placeholders stripped), so tests stay structurally in sync with the real template without manual drift.
-- `setupConfigPanel(headers?)` — sets `document.body.innerHTML` and populates all tag containers.
-- `setupWithSelections(opts)` — calls `setupConfigPanel` then pre-selects values via `applyPreset`.
-
-The `__tests__/helpers/` directory is excluded from test discovery via `testPathIgnorePatterns` in `jest.config.cjs`.
-
 **Coverage:** Run `npm run test:coverage` to collect coverage and enforce per-file thresholds. Coverage is opt-in — the pre-commit hook runs `jest --bail` without `--coverage`.
 
 Two files are excluded from coverage collection entirely:
@@ -213,15 +210,7 @@ See `docs/plans/2026-02-18-testing-coverage-design.md` for full rationale.
 
 ### Tool 4 — Spreadsheet Column Requirements
 
-`runBatchAI` maps column headers by name. The active sheet must contain these exact headers (case-sensitive):
-
-| Config key | Column header |
-| --- | --- |
-| `SOURCE_DRIVE` | `source_drive` |
-| `SOURCE_TEXT` | `source_text` |
-| `SYS_PROMPT` | `system_prompt` |
-| `USER_PROMPT` | `user_prompt` |
-| `OUTPUT` | `ai_inference` |
+`runBatchAI` maps column headers by name via `RunConfig` (user-selected in the sidebar — no hardcoded column names). The user selects which columns serve as user prompt inputs, drive file inputs, system prompt, and output. `runBatchAI` calls `resolveColumns` to locate them by header string and `findOrCreateColumn` to create the output column if absent.
 
 The Gemini API key must be set as a Script Property (`GEMINI_API_KEY`) in Apps Script > Project Settings > Script Properties before Tool 4 will run.
 

CLAUDE.md

@@ -0,0 +1,84 @@
+# Contributing
+
+## Branch Workflow
+
+```
+feature-branch → develop   (PR + code review)
+develop        → main      (PR = release gate)
+```
+
+Feature work happens on branches, merged to `develop` via PR. When ready to ship, `develop` is merged to `main` via a PR containing manual QA instructions — that merge is the release gate.
+
+## Adding Features
+
+### Adding a new Gemini tool
+
+See [Tool System](docs/architecture.md#tool-system) in the architecture docs — it's a three-file change.
+
+### Adding a recipe
+
+Recipes are defined in `src/client/recipes.ts` as entries in the `RECIPES` array. Each `RecipeDefinition` describes the recipe's display metadata, the form fields shown during prep, and how those fields map to a `RunConfig` passed to Run AI. Adding a recipe is entirely client-side and requires no server changes — it's one of the most accessible contributions to make.
+
+### Exposing a new server function
+
+See [Build Pipeline](docs/architecture.md#build-pipeline) in the architecture docs — you must both export from `index.ts` and add a footer stub in `rollup.config.js`. If the function is callable from the client, also update `src/client/google.d.ts`.
+
+## Testing
+
+Tests live in `__tests__/`. Run them with:
+
+```bash
+npm test                    # all tests
+npm run test:watch          # watch mode
+npm run test:coverage       # with per-file coverage thresholds
+```
+
+### Mocking GAS globals
+
+Apps Script globals (`UrlFetchApp`, `DriveApp`, `SpreadsheetApp`, etc.) must be set on `globalThis` **before** importing the module under test, because imports execute immediately:
+
+```ts
+(globalThis as any).UrlFetchApp = { fetch: jest.fn() };
+const { callGeminiAPI } = await import("../src/server/api");
+```
+
+### Mocking `google.script.run`
+
+Capture the success/failure handlers registered by the function under test, then invoke them manually to simulate GAS callbacks:
+
+```ts
+const mockRun = {
+  withSuccessHandler: jest.fn().mockReturnThis(),
+  withFailureHandler: jest.fn().mockReturnThis(),
+  myServerFunction: jest.fn(),
+};
+(globalThis as unknown as { google: unknown }).google = { script: { run: mockRun } };
+
+let capturedSuccess: (v: unknown) => void;
+mockRun.withSuccessHandler.mockImplementation((fn) => {
+  capturedSuccess = fn;
+  return mockRun;
+});
+// Later: capturedSuccess(mockValue) to simulate a successful GAS response.
+```
+
+### Coverage
+
+Coverage is enforced per-file. Run `npm run test:coverage` to check thresholds. Two files are excluded from coverage collection:
+
+- `src/server/index.ts` — deeply coupled to SpreadsheetApp UI globals, not unit-tested.
+- `src/client/sidebar-entry.ts` — calls `init()` immediately at module load time, before `beforeEach` can set up the DOM.
+
+## Code Style
+
+Follows the Google TypeScript Style Guide, enforced by ESLint + Prettier + pre-commit hooks:
+
+- Named exports only (no default exports)
+- `const` by default; no `var`, no `namespace`
+- `===` always; avoid `any` (prefer `unknown`)
+- UpperCamelCase for types/interfaces, lowerCamelCase for functions/variables, CONSTANT_CASE for constants
+- Semicolons required, double quotes, trailing commas
+- Explicit return types on exported functions
+- Prefix unused parameters with `_`
+
+Run `npm run lint:fix` and `npm run format` before pushing.

CONTRIBUTING.md

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ProPublica
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.