diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6b51f05..47b78fd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -26,6 +26,9 @@ jobs: - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 with: bun-version: 1.4.0 + - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 + with: + node-version: 24.7.0 - run: bun install --frozen-lockfile - run: bun run audit - run: bun run fmt:check diff --git a/README.md b/README.md index 29cec12..7921f5f 100644 --- a/README.md +++ b/README.md @@ -3,16 +3,22 @@ Private-first Apache-2.0 workspace for the Ask Gina programmatic client, Bun CLI, listed-plugin portable core, host adapters, and hermetic evals. -This repository is not public and does not publish packages. Pull-request CI is credential-free and build-only. Production Gina MCP remains at `https://askgina.ai/ai/gina/mcp`. -Callers supply a bearer token. The client exposes only the 29 catalog read tools. +This repository is not public and does not publish packages. Pull-request CI runs +workflow gates, but `main` has no protected required checks. Production Gina MCP +remains at `https://askgina.ai/ai/gina/mcp`. Callers supply a bearer token. The +client exposes only the 29 catalog read tools. -## Packages +## Packages and runtimes -- `@askgina/contracts` — public catalog, protocol literals, and receipt schemas -- `@askgina/sdk` — TypeScript client -- `@askgina/cli` — Bun `ask-gina` binary -- `@askgina/plugin-core` — portable plugin source and loaders -- `@askgina/evals` — shared hermetic/live eval contracts, adapters, replay, grading, and sanitization +- `@askgina/contracts` — public catalog, protocol literals, and receipt schemas; + root-only ESM for Node >=24 and Bun >=1.4 +- `@askgina/sdk` — TypeScript client; root-only ESM for Node >=24 and Bun >=1.4 +- `@askgina/cli` — compiled Bun `ask-gina` binary; Bun 1.4.x only +- `@askgina/plugin-core` — host-specific plugin core and loaders +- `@askgina/evals` — compiled hermetic/live eval tools, shared contracts, + adapters, replay, grading, and sanitization; Bun 1.4.x only + +The contracts and SDK packages have no CommonJS, browser, edge, or subpath entrypoints. ## Commands @@ -30,7 +36,9 @@ bun run check:public-boundary bun run smoke:install ``` -`bun run artifacts` emits five package tarballs, five complete host archives, -one four-skill candidate archive, and contract, package, target, and eval receipts -under ignored `dist/`. Nothing in this repository publishes, releases, deploys, +`bun run artifacts` builds the five package outputs with `vp pack`; the +custom packer then creates five package tarballs, five complete host archives, one +four-skill candidate archive, and contract, package, target, and eval receipts +under ignored `dist/`. `bun run verify:artifacts` performs clean +tarball installs and runtime checks. Nothing here publishes, releases, deploys, submits, or calls production during pull-request CI. diff --git a/docs/architecture.md b/docs/architecture.md index e11fbe4..a777c2e 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -10,6 +10,11 @@ The dependency DAG is `contracts -> sdk/plugin-core -> cli/evals`. Workspace cycles are forbidden, and packed artifacts replace workspace ranges with the single repository release version before clean-install verification. +`@askgina/contracts` and `@askgina/sdk` expose only their ESM package roots: no +CommonJS, browser, edge, or subpath entrypoints. They support Node >=24 and Bun + +> =1.4. The compiled CLI and eval executables run on Bun 1.4.x only. + ## Skills `plugins/ask-gina/skills/` is the only authoring source. Pack-time generation @@ -25,22 +30,30 @@ Library and tool APIs return typed Effects. Promise APIs are confined to externa integration boundaries, and application executables provide Bun services once. Live evals share one suite, observation, grading, replay, and sanitized-report pipeline. Responses API and Codex CLI adapters only translate host evidence into -that common contract; they cannot change the rubric. +that common contract; they cannot change the rubric. Eval commands execute the +compiled binaries in `dist/`; suite definitions and observation fixtures remain +repository YAML inputs. + The patched Effect TSGo compiler and official Vite+ Oxlint preset fail on every enabled compiler and Effect diagnostic; there is no baseline or count ratchet. ## Artifacts -The pack task creates five npm-style package archives, five host archives, one -skills candidate, and four bounded receipts. Verification compares hashes and -file lists, performs fresh package-closure installs, reruns target conformance, -and regenerates the four-dimension hermetic eval aggregate from synthetic files. -`dist/` is generated evidence, never authoring source. +`vp pack` writes each package's ignored compiled JavaScript, declarations, and +source maps; every map embeds committed TypeScript through relative paths. The +repository's custom packer stages those validated outputs with metadata and assets +as five npm-style package archives, five host archives, one skills candidate, and +four bounded receipts under ignored `dist/`. Verification clean-installs package +tarballs in temporary projects, checks their closures, runs their actual runtime +entrypoints, reruns target conformance, and regenerates the four-dimension +hermetic eval aggregate from synthetic files. Generated `dist/` content is evidence, never authoring source. ## CI -GitHub Actions are fork-safe and build-only. Pull-request workflows use -`pull_request` with `contents: read`; dependency audit and package clean-install -smoke checks are blocking. A separate manual Responses live-smoke workflow may -read two protected secrets and uploads only the sanitized aggregate. There is no -release job, OIDC, package writer, or remote cache. +Fork-safe GitHub Actions run formatting, lint, compiler, test, audit, target +conformance, artifact clean-install/runtime, and public-boundary gates on pull +requests. These workflows use `pull_request` with `contents: read`, but `main` +has no protected required checks, so they do not gate merge. A separate manual +Responses live-smoke workflow may read two protected secrets and uploads only the +sanitized aggregate. There is no release job, OIDC, package writer, remote cache, +or publication authority. diff --git a/docs/contributing.md b/docs/contributing.md index aaa3e9d..a8a0a74 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -4,14 +4,14 @@ Edit canonical skill source only under `plugins/ask-gina/skills/`. Host skill trees are generated at pack time and must not be committed. Keep the workspace on Bun 1.4.0, Vite+ 0.3.0, Vitest 4.1.11, and Effect -4.0.0-rc.111. Dependency updates are reviewed plan revisions, not drive-by -range bumps. +4.0.0-rc.111. The contracts and SDK artifacts also support Node >=24. Dependency +updates are reviewed plan revisions, not drive-by range bumps. Do not add secrets, OIDC, package write, remote cache, or release workflows. Public contract changes must not include `factoryKey`, JWT material, private hosts, or authenticated observations. -Run the full local gate before proposing a change: +Use the repository scripts before proposing a change: ```sh bun install --frozen-lockfile @@ -25,6 +25,12 @@ bun run verify:artifacts bun run check:public-boundary ``` +`bun run artifacts` builds ignored package outputs with `vp pack` and stages the +custom packer's ignored archives and receipts under repository `dist/`; +`bun run verify:artifacts` checks clean tarball installs and their runtime +entrypoints. Pull-request CI runs these repository gates, but `main` has no +protected required checks, so CI does not gate merge or publish packages. + Do not suppress Effect diagnostics or add diagnostic baselines, report files, or count ratchets. Fix the source. Package manifests, plugin manifests, and host manifests must keep the root release version exactly aligned. diff --git a/docs/programmatic-client.md b/docs/programmatic-client.md index 8734497..3875481 100644 --- a/docs/programmatic-client.md +++ b/docs/programmatic-client.md @@ -1,7 +1,24 @@ # Programmatic client The TypeScript SDK and Bun CLI talk to production Gina MCP at -`https://askgina.ai/ai/gina/mcp` with a caller-supplied bearer. +`https://askgina.ai/ai/gina/mcp` with a caller-supplied bearer token. Install and +consume the packed artifacts; do not import files from the workspace `src/` +directories. This repository does not publish the packages. + +## Runtime and package contract + +- `@askgina/contracts` and `@askgina/sdk` are compiled ESM packages for Node.js + `>=24` and Bun `>=1.4`. Import each package from its root only. +- `@askgina/cli` requires Bun `1.4.x`. Its installed `ask-gina` command points to + the compiled ESM bin at `./dist/bin.js`. +- CommonJS, browser runtimes, edge runtimes, and package subpaths are not + supported. + +`vp pack` writes each package's compiled JavaScript, declarations, and maps to +its ignored `dist/` directory. The maps embed the committed TypeScript and use +relative source paths. + +## SDK usage ```ts import { createClient } from "@askgina/sdk"; @@ -9,6 +26,8 @@ import { createClient } from "@askgina/sdk"; const client = createClient({ accessToken: process.env.ASK_GINA_ACCESS_TOKEN ?? "" }); ``` +## CLI usage + ```sh export ASK_GINA_ACCESS_TOKEN=synthetic-fixture ask-gina list diff --git a/docs/security.md b/docs/security.md index fec8842..758da42 100644 --- a/docs/security.md +++ b/docs/security.md @@ -21,6 +21,12 @@ Forbidden in source, packages, archives, and receipts: archives. It fails on private imports or hosts, credential-like values, unsafe or unexpected archive content, private runtime dependencies, and raw eval fields. The check has no credentialed mode and no release override. +`vp pack` writes ignored compiled package output. Custom artifact +verification clean-installs the tarballs and runs their compiled entrypoints. +Contracts and SDK expose only root ESM imports for Node >=24 and Bun >=1.4. +CLI and eval executables require Bun 1.4.x. No package supports CommonJS, +browser, edge, or subpath entrypoints. CI runs these checks, but `main` has no +protected required checks. Nothing here grants publication authority. Live eval credentials are read from environment-backed Effect `Config` values, never command-line arguments. The Responses runner sends the provider key only to the Responses API and the Gina bearer only in the remote MCP authorization diff --git a/package.json b/package.json index b2af36a..43d1c02 100644 --- a/package.json +++ b/package.json @@ -10,23 +10,25 @@ "type": "module", "scripts": { "fmt:check": "vp fmt --check .", - "lint": "vp lint --deny-warnings .", + "lint": "bun run build && vp lint --deny-warnings .", "check": "vp run quality", + "build": "vp run build-packages", "test": "vp run tests", "artifacts": "vp run build-artifacts", "verify:artifacts": "vp run verify-artifacts", "check:public-boundary": "vp run public-boundary", "check:target-conformance": "vp run target-conformance", + "check:typescript-imports": "bun tools/check-typescript-imports.ts", "verify:packages": "bun tools/verify-artifacts.ts --packages", "verify:targets": "bun tools/verify-artifacts.ts --targets", "smoke:install": "bun tools/verify-artifacts.ts --packages", "audit": "bun audit", - "eval:replay": "bun packages/evals/src/bin/replay.ts", - "eval:responses": "bun packages/evals/src/bin/live.ts --runner responses", - "eval:codex": "bun packages/evals/src/bin/live.ts --runner codex", + "eval:replay": "bun run build && bun packages/evals/dist/bin/replay.js", + "eval:responses": "bun run build && bun packages/evals/dist/bin/live.js --runner responses", + "eval:codex": "bun run build && bun packages/evals/dist/bin/live.js --runner codex", "effect-tsgo:patch": "bun scripts/effect-tsgo-patch-if-needed.ts", "prepare": "bun run effect-tsgo:patch", - "typecheck": "node_modules/.bin/tsc --noEmit -p tsconfig.json", + "typecheck": "bun run build && node_modules/.bin/tsc --noEmit -p tsconfig.json", "effect-tsgo:diagnostics": "node_modules/.bin/effect-tsgo diagnostics --project tsconfig.json --format json" }, "devDependencies": { @@ -47,8 +49,7 @@ "oxlint": "1.78.0" }, "engines": { - "bun": "1.4.x", - "node": "22.x" + "bun": "1.4.x" }, "packageManager": "bun@1.4.0" } diff --git a/packages/cli/README.md b/packages/cli/README.md index e29f202..a441e96 100644 --- a/packages/cli/README.md +++ b/packages/cli/README.md @@ -1,3 +1,19 @@ # @askgina/cli -See repository README. +`@askgina/cli` is a Bun `1.4.x`-only CLI. Install the packed artifact with Bun; +the installed `ask-gina` command points to the compiled ESM bin at +`./dist/bin.js`. Do not run the workspace TypeScript source directly. This +repository does not publish the package. + +```sh +export ASK_GINA_ACCESS_TOKEN=synthetic-fixture +ask-gina list +ask-gina call gina.listScheduledPrompts '{}' +``` + +Node.js, CommonJS, browser runtimes, edge runtimes, and package subpaths are not +supported. + +`vp pack` writes compiled JavaScript, declarations, and maps to the ignored +`dist/` directory. The maps embed the committed TypeScript and use relative +source paths. diff --git a/packages/cli/__tests__/cli.test.ts b/packages/cli/__tests__/cli.test.ts index cfb5884..777f54c 100644 --- a/packages/cli/__tests__/cli.test.ts +++ b/packages/cli/__tests__/cli.test.ts @@ -10,7 +10,7 @@ import { } from "@askgina/sdk"; import { ConfigProvider, Effect } from "effect"; -import { AskGinaCliExitCode, recoverAskGinaCliFailures, runAskGinaCli } from "../src/run.js"; +import { AskGinaCliExitCode, recoverAskGinaCliFailures, runAskGinaCli } from "../src/run"; const mockTransport: AskGinaTransport = { listTools: () => Effect.succeed(listCatalogToolNames().map((name) => ({ name }))), diff --git a/packages/cli/bin.ts b/packages/cli/bin.ts index 0d89bd7..c6d9698 100755 --- a/packages/cli/bin.ts +++ b/packages/cli/bin.ts @@ -4,7 +4,7 @@ import * as BunRuntime from "@effect/platform-bun/BunRuntime"; import * as BunServices from "@effect/platform-bun/BunServices"; import { Effect, Layer } from "effect"; -import { recoverAskGinaCliFailures, runAskGinaCliFromStdio } from "./src/run.js"; +import { recoverAskGinaCliFailures, runAskGinaCliFromStdio } from "./src/run"; const program: Effect.Effect = recoverAskGinaCliFailures( runAskGinaCliFromStdio(), diff --git a/packages/cli/package.json b/packages/cli/package.json index dd8c831..601aaf5 100644 --- a/packages/cli/package.json +++ b/packages/cli/package.json @@ -3,21 +3,31 @@ "version": "0.1.0", "license": "Apache-2.0", "bin": { - "ask-gina": "./bin.ts" + "ask-gina": "./dist/bin.js" }, "files": [ - "bin.ts", - "src", + "dist", "LICENSE", "README.md" ], "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", "exports": { - ".": "./src/index.ts" + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "scripts": { + "build": "vp -C ../.. pack --filter cli" }, "dependencies": { "@askgina/sdk": "workspace:*", "@effect/platform-bun": "4.0.0-rc.111", "effect": "4.0.0-rc.111" + }, + "engines": { + "bun": "1.4.x" } } diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts index 69ebda8..c2acac3 100644 --- a/packages/cli/src/index.ts +++ b/packages/cli/src/index.ts @@ -4,4 +4,4 @@ export { runAskGinaCli, type AskGinaCliOptions, type AskGinaParseFailure, -} from "./run.js"; +} from "./run"; diff --git a/packages/cli/src/run.ts b/packages/cli/src/run.ts index 5331380..76fe018 100644 --- a/packages/cli/src/run.ts +++ b/packages/cli/src/run.ts @@ -2,7 +2,7 @@ import type { AskGinaError, AskGinaTransport } from "@askgina/sdk"; import { Console, Effect } from "effect"; import { CliError, Command } from "effect/unstable/cli"; -import { makeAskGinaCommand } from "./command.js"; +import { makeAskGinaCommand } from "./command"; export type AskGinaCliOptions = { readonly argv: ReadonlyArray; diff --git a/packages/contracts/README.md b/packages/contracts/README.md index 9f9f71a..d8c01bb 100644 --- a/packages/contracts/README.md +++ b/packages/contracts/README.md @@ -1,3 +1,18 @@ # @askgina/contracts -See repository README. +`@askgina/contracts` ships as compiled ESM for Node.js `>=24` and Bun `>=1.4`. +Install and consume the packed artifact rather than importing workspace source +files. This repository does not publish the package. + +Import only from the package root: + +```ts +import { listCatalogToolNames } from "@askgina/contracts"; +``` + +CommonJS, browser runtimes, edge runtimes, and package subpaths such as +`@askgina/contracts/...` are not supported. + +`vp pack` writes compiled JavaScript, declarations, and maps to the ignored +`dist/` directory. The maps embed the committed TypeScript and use relative +source paths. diff --git a/packages/contracts/package.json b/packages/contracts/package.json index 199a9f8..fb8c1d2 100644 --- a/packages/contracts/package.json +++ b/packages/contracts/package.json @@ -3,15 +3,27 @@ "version": "0.1.0", "license": "Apache-2.0", "files": [ - "src", + "dist", "LICENSE", "README.md" ], "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", "exports": { - ".": "./src/index.ts" + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "scripts": { + "build": "vp -C ../.. pack --filter contracts" }, "dependencies": { "effect": "4.0.0-rc.111" + }, + "engines": { + "bun": ">=1.4.0", + "node": ">=24" } } diff --git a/packages/evals/README.md b/packages/evals/README.md index 94a8bf7..7a6292a 100644 --- a/packages/evals/README.md +++ b/packages/evals/README.md @@ -4,6 +4,12 @@ One schema and rubric drive hermetic replay, OpenAI Responses API trials, and Codex CLI trials. Live runners use the same suite cases, model, reasoning mode, case selection, repetition count, and sanitized aggregate shape. +`@askgina/evals` is a Bun 1.4.x-only compiled `dist` package. The root +`eval:replay`, `eval:responses`, and `eval:codex` commands build the package graph, then execute `packages/evals/dist/bin/*.js`; suite and observation YAML remain repository inputs. +Artifact verification clean-installs the built tarball and exercises its compiled +import and replay entrypoint. The package supports only its root ESM import; Node.js, +CommonJS, browser and edge runtimes, and subpath imports are unsupported. + ## Hermetic replay ```sh diff --git a/packages/evals/__tests__/codex-cli.test.ts b/packages/evals/__tests__/codex-cli.test.ts index 2d80aab..7f5e207 100644 --- a/packages/evals/__tests__/codex-cli.test.ts +++ b/packages/evals/__tests__/codex-cli.test.ts @@ -21,9 +21,9 @@ import { PluginEvalCodexCliSpawnError, PluginEvalCodexCliTimeoutError, runCodexCliPluginEvalTrial, -} from "../src/codex-cli.js"; -import { collectBoundedUtf8Output } from "../src/bounded-output.js"; -import type { PluginEvalCase } from "../src/contracts.js"; +} from "../src/codex-cli"; +import { collectBoundedUtf8Output } from "../src/bounded-output"; +import type { PluginEvalCase } from "../src/contracts"; const ASK_GINA_SKILL_PATH = "/home/eval/.codex/plugins/cache/personal/ask-gina/0.1.0/skills/research-spot-tokens/SKILL.md"; diff --git a/packages/evals/__tests__/responses-api.test.ts b/packages/evals/__tests__/responses-api.test.ts index b7c5054..392f6b6 100644 --- a/packages/evals/__tests__/responses-api.test.ts +++ b/packages/evals/__tests__/responses-api.test.ts @@ -6,7 +6,7 @@ import * as HttpClientError from "effect/unstable/http/HttpClientError"; import type * as HttpClientRequest from "effect/unstable/http/HttpClientRequest"; import * as HttpClientResponse from "effect/unstable/http/HttpClientResponse"; -import type { PluginEvalCase } from "../src/contracts.js"; +import type { PluginEvalCase } from "../src/contracts"; import { PluginEvalResponsesDecodeError, PluginEvalResponsesHttpError, @@ -15,7 +15,7 @@ import { RESPONSES_API_MAX_BODY_BYTES, type ResponsesApiTrialOptions, runResponsesApiPluginEvalTrial, -} from "../src/responses-api.js"; +} from "../src/responses-api"; const evalCase: PluginEvalCase = { id: "direct-price", diff --git a/packages/evals/__tests__/runner.test.ts b/packages/evals/__tests__/runner.test.ts index 0788957..b24a46a 100644 --- a/packages/evals/__tests__/runner.test.ts +++ b/packages/evals/__tests__/runner.test.ts @@ -14,8 +14,8 @@ import { replayPluginEvalObservationSet, runLiveEvalSuite, runHermeticEvalReplay, -} from "../src/index.js"; -import { makeSanitizedEvalRunReport } from "../src/report.js"; +} from "../src/index"; +import { makeSanitizedEvalRunReport } from "../src/report"; const collectPublicStrings = (value: unknown): readonly string[] => { if (typeof value === "string") return [value]; diff --git a/packages/evals/__tests__/sanitize.test.ts b/packages/evals/__tests__/sanitize.test.ts index 6045aca..4213823 100644 --- a/packages/evals/__tests__/sanitize.test.ts +++ b/packages/evals/__tests__/sanitize.test.ts @@ -6,7 +6,7 @@ import { sanitizeEvalAggregate, type SanitizedEvalAggregate, type SanitizedEvalAggregateProvenance, -} from "../src/index.js"; +} from "../src/index"; const collectPublicStrings = (value: unknown): readonly string[] => { if (typeof value === "string") return [value]; diff --git a/packages/evals/package.json b/packages/evals/package.json index 23c48bb..395aa31 100644 --- a/packages/evals/package.json +++ b/packages/evals/package.json @@ -3,13 +3,21 @@ "version": "0.1.0", "license": "Apache-2.0", "files": [ - "src", + "dist", "LICENSE", "README.md" ], "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", "exports": { - ".": "./src/index.ts" + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "scripts": { + "build": "vp -C ../.. pack --filter evals" }, "dependencies": { "@askgina/contracts": "workspace:*", @@ -18,5 +26,8 @@ "@effect/platform-bun": "4.0.0-rc.111", "effect": "4.0.0-rc.111", "yaml": "2.8.3" + }, + "engines": { + "bun": "1.4.x" } } diff --git a/packages/evals/src/bin/live.ts b/packages/evals/src/bin/live.ts index 46b1a04..31f3886 100755 --- a/packages/evals/src/bin/live.ts +++ b/packages/evals/src/bin/live.ts @@ -32,17 +32,17 @@ import { openAttestedCodexExecutable, runCodexCliPluginEvalTrial, type AttestedCodexExecutable, -} from "../codex-cli.js"; -import { collectBoundedUtf8Output } from "../bounded-output.js"; -import { loadPluginEvalSuite } from "../load-suite.js"; +} from "../codex-cli"; +import { collectBoundedUtf8Output } from "../bounded-output"; +import { loadPluginEvalSuite } from "../load-suite"; import { MAXIMUM_LIVE_REPETITIONS, MINIMUM_LIVE_REPETITIONS, preflightLiveEvalSuite, runLiveEvalSuite, -} from "../live.js"; -import { runResponsesApiPluginEvalTrial } from "../responses-api.js"; -import type { SanitizedEvalRunReport } from "../report.js"; +} from "../live"; +import { runResponsesApiPluginEvalTrial } from "../responses-api"; +import type { SanitizedEvalRunReport } from "../report"; const GIT_STATUS_LIMIT_BYTES = 65_536; const CODEX_PREFLIGHT_LIMIT_BYTES = 1_048_576; diff --git a/packages/evals/src/bin/replay.ts b/packages/evals/src/bin/replay.ts index 58fccbd..463525b 100755 --- a/packages/evals/src/bin/replay.ts +++ b/packages/evals/src/bin/replay.ts @@ -4,8 +4,8 @@ import * as BunRuntime from "@effect/platform-bun/BunRuntime"; import * as BunServices from "@effect/platform-bun/BunServices"; import { Console, Data, Effect, FileSystem, Layer, Path, Schema } from "effect"; -import { makeSanitizedEvalRunReport } from "../report.js"; -import { runHermeticEvalReplay } from "../runner.js"; +import { makeSanitizedEvalRunReport } from "../report"; +import { runHermeticEvalReplay } from "../runner"; interface ReplayCliOptions { readonly suitePath: string; diff --git a/packages/evals/src/codex-cli.ts b/packages/evals/src/codex-cli.ts index a370a25..80e502f 100644 --- a/packages/evals/src/codex-cli.ts +++ b/packages/evals/src/codex-cli.ts @@ -22,8 +22,8 @@ import type { PluginEvalObservation, PluginEvalTokenUsage, PluginEvalToolCall, -} from "./contracts.js"; -import { collectBoundedUtf8Output } from "./bounded-output.js"; +} from "./contracts"; +import { collectBoundedUtf8Output } from "./bounded-output"; const DEFAULT_TIMEOUT_MS = 120_000; const DEFAULT_MODEL = "codex-cli"; diff --git a/packages/evals/src/grading.ts b/packages/evals/src/grading.ts index a72658d..95dd638 100644 --- a/packages/evals/src/grading.ts +++ b/packages/evals/src/grading.ts @@ -6,7 +6,7 @@ import type { PluginEvalDimensionScore, PluginEvalObservation, PluginEvalToolCall, -} from "./contracts.js"; +} from "./contracts"; export class PluginEvalObservationMismatchError extends Data.TaggedError( "PluginEvalObservationMismatchError", diff --git a/packages/evals/src/index.ts b/packages/evals/src/index.ts index b84e3b3..4124698 100644 --- a/packages/evals/src/index.ts +++ b/packages/evals/src/index.ts @@ -4,28 +4,28 @@ export { PluginEvalObservationSetParseError, PluginEvalObservationSetReadError, PluginEvalObservationSetValidationError, -} from "./load-observations.js"; +} from "./load-observations"; export { decodePluginEvalSuite, loadPluginEvalSuite, PluginEvalSuiteParseError, PluginEvalSuiteReadError, PluginEvalSuiteValidationError, -} from "./load-suite.js"; -export { gradePluginEvalObservation, PluginEvalObservationMismatchError } from "./grading.js"; -export { replayPluginEvalObservationSet, PluginEvalReplayContractError } from "./replay.js"; +} from "./load-suite"; +export { gradePluginEvalObservation, PluginEvalObservationMismatchError } from "./grading"; +export { replayPluginEvalObservationSet, PluginEvalReplayContractError } from "./replay"; export { runHermeticEvalReplay, type HermeticEvalReplayError, type HermeticEvalReplayOptions, type HermeticEvalReplayResult, -} from "./runner.js"; +} from "./runner"; export { LiveEvalSelectionError, runLiveEvalSuite, type LiveEvalOptions, type LiveEvalTrialInput, -} from "./live.js"; +} from "./live"; export { PluginEvalResponsesDecodeError, PluginEvalResponsesHttpError, @@ -34,7 +34,7 @@ export { runResponsesApiPluginEvalTrial, type PluginEvalResponsesError, type ResponsesApiTrialOptions, -} from "./responses-api.js"; +} from "./responses-api"; export { attestCodexExecutable, PluginEvalCodexCliExecutableError, @@ -49,7 +49,7 @@ export { type CodexCliTrialOptions, type CodexCliTrialRunner, type PluginEvalCodexCliError, -} from "./codex-cli.js"; +} from "./codex-cli"; export { makeSanitizedEvalRunReport, sanitizeEvalReplay, @@ -57,7 +57,7 @@ export { SanitizedEvalRunReportSchema, type EvalReplayProvenance, type SanitizedEvalRunReport, -} from "./report.js"; +} from "./report"; export { ALLOWED_SYNTHETIC_FIXTURE_PROMPTS, findPublicTextViolations, @@ -73,7 +73,7 @@ export { type SanitizedEvalDimensionName, type SanitizedEvalDimensionSummary, type SanitizedEvalDistribution, -} from "./sanitize.js"; +} from "./sanitize"; export { PluginEvalCaseSchema, PluginEvalCaseScoreSchema, @@ -96,4 +96,4 @@ export { type PluginEvalReplayReport, type PluginEvalSuite, type PluginEvalToolCall, -} from "./contracts.js"; +} from "./contracts"; diff --git a/packages/evals/src/live.ts b/packages/evals/src/live.ts index aede927..35a6b96 100644 --- a/packages/evals/src/live.ts +++ b/packages/evals/src/live.ts @@ -1,20 +1,20 @@ import { catalogSha, isGinaReadToolName, listCatalogToolNames } from "@askgina/contracts"; import { Data, DateTime, Effect, Function } from "effect"; -import type { PluginEvalCase, PluginEvalObservation, PluginEvalSuite } from "./contracts.js"; -import type { PluginEvalObservationMismatchError } from "./grading.js"; +import type { PluginEvalCase, PluginEvalObservation, PluginEvalSuite } from "./contracts"; +import type { PluginEvalObservationMismatchError } from "./grading"; import { decodePluginEvalObservationSet, type PluginEvalObservationSetValidationError, -} from "./load-observations.js"; -import { replayPluginEvalObservationSet, type PluginEvalReplayContractError } from "./replay.js"; +} from "./load-observations"; +import { replayPluginEvalObservationSet, type PluginEvalReplayContractError } from "./replay"; import { assertSanitizedRunMetadata, makeSanitizedEvalRunReport, type SanitizedEvalRunReport, type SanitizedEvalRunReportError, -} from "./report.js"; -import type { HermeticEvalSanitizationError } from "./sanitize.js"; +} from "./report"; +import type { HermeticEvalSanitizationError } from "./sanitize"; export const MINIMUM_LIVE_REPETITIONS = 3; export const MAXIMUM_LIVE_REPETITIONS = 5; export const MAXIMUM_LIVE_CASES = 64; diff --git a/packages/evals/src/load-observations.ts b/packages/evals/src/load-observations.ts index 8c283c4..1abb873 100644 --- a/packages/evals/src/load-observations.ts +++ b/packages/evals/src/load-observations.ts @@ -1,7 +1,7 @@ import { Data, Effect, FileSystem, Function, Path, Schema } from "effect"; import { parse as parseYaml } from "yaml"; -import { type PluginEvalObservationSet, PluginEvalObservationSetSchema } from "./contracts.js"; +import { type PluginEvalObservationSet, PluginEvalObservationSetSchema } from "./contracts"; export class PluginEvalObservationSetReadError extends Data.TaggedError( "PluginEvalObservationSetReadError", diff --git a/packages/evals/src/load-suite.ts b/packages/evals/src/load-suite.ts index 9d6901a..ae114b3 100644 --- a/packages/evals/src/load-suite.ts +++ b/packages/evals/src/load-suite.ts @@ -1,7 +1,7 @@ import { Data, Effect, FileSystem, Function, Path, Schema } from "effect"; import { parse as parseYaml } from "yaml"; -import { type PluginEvalCase, type PluginEvalSuite, PluginEvalSuiteSchema } from "./contracts.js"; +import { type PluginEvalCase, type PluginEvalSuite, PluginEvalSuiteSchema } from "./contracts"; export class PluginEvalSuiteReadError extends Data.TaggedError("PluginEvalSuiteReadError")<{ readonly path: string; diff --git a/packages/evals/src/replay.ts b/packages/evals/src/replay.ts index a7441b3..d96d3f7 100644 --- a/packages/evals/src/replay.ts +++ b/packages/evals/src/replay.ts @@ -10,8 +10,8 @@ import type { PluginEvalReplayReport, PluginEvalTokenUsageSummary, PluginEvalSuite, -} from "./contracts.js"; -import { gradePluginEvalObservation, type PluginEvalObservationMismatchError } from "./grading.js"; +} from "./contracts"; +import { gradePluginEvalObservation, type PluginEvalObservationMismatchError } from "./grading"; export class PluginEvalReplayContractError extends Data.TaggedError( "PluginEvalReplayContractError", diff --git a/packages/evals/src/report.ts b/packages/evals/src/report.ts index 8961757..ed60fee 100644 --- a/packages/evals/src/report.ts +++ b/packages/evals/src/report.ts @@ -1,12 +1,12 @@ import { Data, Effect, Schema } from "effect"; -import type { PluginEvalReplayReport, PluginEvalRunManifest } from "./contracts.js"; +import type { PluginEvalReplayReport, PluginEvalRunManifest } from "./contracts"; import { findPublicTextViolations, HermeticEvalSanitizationError, sanitizeEvalAggregate, SanitizedEvalAggregateSchema, -} from "./sanitize.js"; +} from "./sanitize"; const SafeRunLabelSchema = Schema.NonEmptyString.check( Schema.isMaxLength(128), diff --git a/packages/evals/src/responses-api.ts b/packages/evals/src/responses-api.ts index cb45266..67ce118 100644 --- a/packages/evals/src/responses-api.ts +++ b/packages/evals/src/responses-api.ts @@ -3,8 +3,8 @@ import { Clock, Data, DateTime, Duration, Effect, Function, Schema } from "effec import * as HttpClient from "effect/unstable/http/HttpClient"; import * as HttpClientRequest from "effect/unstable/http/HttpClientRequest"; -import type { PluginEvalCase, PluginEvalObservation, PluginEvalToolCall } from "./contracts.js"; -import { collectBoundedUtf8Output } from "./bounded-output.js"; +import type { PluginEvalCase, PluginEvalObservation, PluginEvalToolCall } from "./contracts"; +import { collectBoundedUtf8Output } from "./bounded-output"; const OPENAI_RESPONSES_URL = "https://api.openai.com/v1/responses"; const DEFAULT_TIMEOUT_MS = 120_000; diff --git a/packages/evals/src/runner.ts b/packages/evals/src/runner.ts index 846ccb6..c329ae0 100644 --- a/packages/evals/src/runner.ts +++ b/packages/evals/src/runner.ts @@ -1,21 +1,21 @@ import { catalogSha } from "@askgina/contracts"; import { Effect, FileSystem, Path } from "effect"; -import type { PluginEvalReplayReport, PluginEvalRunManifest } from "./contracts.js"; +import type { PluginEvalReplayReport, PluginEvalRunManifest } from "./contracts"; import { loadPluginEvalObservationSet, type PluginEvalObservationSetParseError, type PluginEvalObservationSetReadError, type PluginEvalObservationSetValidationError, -} from "./load-observations.js"; +} from "./load-observations"; import { loadPluginEvalSuite, type PluginEvalSuiteParseError, type PluginEvalSuiteReadError, type PluginEvalSuiteValidationError, -} from "./load-suite.js"; -import type { PluginEvalObservationMismatchError } from "./grading.js"; -import { replayPluginEvalObservationSet, type PluginEvalReplayContractError } from "./replay.js"; +} from "./load-suite"; +import type { PluginEvalObservationMismatchError } from "./grading"; +import { replayPluginEvalObservationSet, type PluginEvalReplayContractError } from "./replay"; export interface HermeticEvalReplayOptions { readonly suitePath: string; diff --git a/packages/evals/src/sanitize.ts b/packages/evals/src/sanitize.ts index 33cd889..774b693 100644 --- a/packages/evals/src/sanitize.ts +++ b/packages/evals/src/sanitize.ts @@ -1,6 +1,6 @@ import { Data, Effect, Function, Schema } from "effect"; -import { isUnknownRecord } from "./type-guards.js"; +import { isUnknownRecord } from "./type-guards"; export const ALLOWED_SYNTHETIC_FIXTURE_PROMPTS = [ "Look up the synthetic label amber.", diff --git a/packages/sdk/README.md b/packages/sdk/README.md index caf3010..6f240e1 100644 --- a/packages/sdk/README.md +++ b/packages/sdk/README.md @@ -1,3 +1,20 @@ # @askgina/sdk -See repository README. +`@askgina/sdk` ships as compiled ESM for Node.js `>=24` and Bun `>=1.4`. +Install and consume the packed artifact rather than importing workspace source +files. This repository does not publish the package. + +Import only from the package root: + +```ts +import { createClient } from "@askgina/sdk"; + +const client = createClient({ accessToken: process.env.ASK_GINA_ACCESS_TOKEN ?? "" }); +``` + +CommonJS, browser runtimes, edge runtimes, and package subpaths such as +`@askgina/sdk/...` are not supported. + +`vp pack` writes compiled JavaScript, declarations, and maps to the ignored +`dist/` directory. The maps embed the committed TypeScript and use relative +source paths. diff --git a/packages/sdk/package.json b/packages/sdk/package.json index e2f4053..8ccaecd 100644 --- a/packages/sdk/package.json +++ b/packages/sdk/package.json @@ -3,17 +3,29 @@ "version": "0.1.0", "license": "Apache-2.0", "files": [ - "src", + "dist", "LICENSE", "README.md" ], "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", "exports": { - ".": "./src/index.ts" + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "scripts": { + "build": "vp -C ../.. pack --filter sdk" }, "dependencies": { "@askgina/contracts": "workspace:*", "@modelcontextprotocol/sdk": "1.29.0", "effect": "4.0.0-rc.111" + }, + "engines": { + "bun": ">=1.4.0", + "node": ">=24" } } diff --git a/packages/sdk/src/__tests__/client.test.ts b/packages/sdk/src/__tests__/client.test.ts index 8dd4a3f..9b76100 100644 --- a/packages/sdk/src/__tests__/client.test.ts +++ b/packages/sdk/src/__tests__/client.test.ts @@ -5,9 +5,9 @@ import { assert, describe, it } from "@effect/vitest"; import { Effect, Result } from "effect"; import { beforeEach, vi } from "vitest"; -import { createClient, listCatalogToolNames } from "../client.js"; -import { AskGinaAuthError, AskGinaToolError } from "../errors.js"; -import type { AskGinaTransport } from "../transport.js"; +import { createClient, listCatalogToolNames } from "../client"; +import { AskGinaAuthError, AskGinaToolError } from "../errors"; +import type { AskGinaTransport } from "../transport"; const mcpMocks = vi.hoisted(() => ({ callTool: vi.fn<() => Promise>(), clientConstructor: vi.fn<() => void>(), diff --git a/packages/sdk/src/__tests__/transport.test.ts b/packages/sdk/src/__tests__/transport.test.ts index a0931db..601792b 100644 --- a/packages/sdk/src/__tests__/transport.test.ts +++ b/packages/sdk/src/__tests__/transport.test.ts @@ -5,8 +5,8 @@ import { assert, describe, it } from "@effect/vitest"; import { Effect, Fiber, Result } from "effect"; import { beforeEach, vi } from "vitest"; -import { AskGinaToolError, AskGinaTransportError } from "../errors.js"; -import { createProductionTransport, rejectIfMcpToolError } from "../transport.js"; +import { AskGinaToolError, AskGinaTransportError } from "../errors"; +import { createProductionTransport, rejectIfMcpToolError } from "../transport"; type RequestOptions = { readonly signal?: AbortSignal; diff --git a/packages/sdk/src/client.ts b/packages/sdk/src/client.ts index 457ef39..057334c 100644 --- a/packages/sdk/src/client.ts +++ b/packages/sdk/src/client.ts @@ -6,10 +6,10 @@ import { } from "@askgina/contracts"; import { Effect } from "effect"; -import type { AskGinaError } from "./errors.js"; -import { AskGinaAuthError, AskGinaToolError } from "./errors.js"; -import type { AskGinaListedTool, AskGinaTransport } from "./transport.js"; -import { createProductionTransport } from "./transport.js"; +import type { AskGinaError } from "./errors"; +import { AskGinaAuthError, AskGinaToolError } from "./errors"; +import type { AskGinaListedTool, AskGinaTransport } from "./transport"; +import { createProductionTransport } from "./transport"; export type AskGinaClientOptions = { readonly accessToken: string; diff --git a/packages/sdk/src/index.ts b/packages/sdk/src/index.ts index 5f0989e..fda746c 100644 --- a/packages/sdk/src/index.ts +++ b/packages/sdk/src/index.ts @@ -1,14 +1,10 @@ -export { createClient, listCatalogToolNames } from "./client.js"; -export type { AskGinaClient, AskGinaClientOptions } from "./client.js"; +export { createClient, listCatalogToolNames } from "./client"; +export type { AskGinaClient, AskGinaClientOptions } from "./client"; export { AskGinaAuthError, AskGinaJsonArgsError, AskGinaToolError, AskGinaTransportError, -} from "./errors.js"; -export type { AskGinaError } from "./errors.js"; -export { - rejectIfMcpToolError, - type AskGinaListedTool, - type AskGinaTransport, -} from "./transport.js"; +} from "./errors"; +export type { AskGinaError } from "./errors"; +export { rejectIfMcpToolError, type AskGinaListedTool, type AskGinaTransport } from "./transport"; diff --git a/packages/sdk/src/transport.ts b/packages/sdk/src/transport.ts index 2f2dff6..b43e057 100644 --- a/packages/sdk/src/transport.ts +++ b/packages/sdk/src/transport.ts @@ -3,8 +3,8 @@ import { Client as McpClient } from "@modelcontextprotocol/sdk/client/index.js"; import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js"; import { Effect, Function } from "effect"; -import type { AskGinaError } from "./errors.js"; -import { AskGinaToolError, AskGinaTransportError } from "./errors.js"; +import type { AskGinaError } from "./errors"; +import { AskGinaToolError, AskGinaTransportError } from "./errors"; export type AskGinaListedTool = { readonly name: string; diff --git a/plugins/ask-gina/README.md b/plugins/ask-gina/README.md index 05c5a50..75097ae 100644 --- a/plugins/ask-gina/README.md +++ b/plugins/ask-gina/README.md @@ -1,3 +1,11 @@ # @askgina/plugin-core -See repository README. +Committed `src/**`, `plugin.yaml`, `skills/**`, `evals/**`, and `targets/**` are the +canonical plugin inputs. `vp pack` derives the package `dist/` and the custom +packer creates complete host archives under the repository's ignored `dist/` +tree; generated package and host outputs are not source and are never +authoritative. + +The repository may build, verify, and archive these artifacts. It does not publish, +release, deploy, or submit them: packaging creates evidence, not publication +authority. diff --git a/plugins/ask-gina/__tests__/plugin-core.test.ts b/plugins/ask-gina/__tests__/plugin-core.test.ts index b01e69d..de8af66 100644 --- a/plugins/ask-gina/__tests__/plugin-core.test.ts +++ b/plugins/ask-gina/__tests__/plugin-core.test.ts @@ -6,9 +6,12 @@ import { assert, describe, it } from "@effect/vitest"; import { ASK_GINA_SKILL_DEFINITIONS, PRODUCTION_MCP_URL, READ_SCOPE } from "@askgina/contracts"; import { Effect, FileSystem, Path, PlatformError, Schema } from "effect"; -import { loadCanonicalSkillDocuments, loadPluginManifest } from "../src/index.js"; -import { createGeneratedPluginTargets, TARGET_NAMES } from "../../../tools/sync-plugin-skills.js"; -import { runTargetConformanceChecks } from "../../../tools/check-target-conformance.js"; +import { loadCanonicalSkillDocuments, loadPluginManifest } from "../src/index"; +import { createGeneratedPluginTargets, TARGET_NAMES } from "../../../tools/sync-plugin-skills"; +import { + checkGeneratedTargetConformance, + runTargetConformanceChecks, +} from "../../../tools/check-target-conformance"; const pluginRoot = fileURLToPath(new URL("../", import.meta.url)); const repositoryRoot = fileURLToPath(new URL("../../../", import.meta.url)); @@ -189,5 +192,31 @@ describe("Ask Gina portable plugin core", () => { assert.strictEqual(Object.keys(report.targets).length, TARGET_NAMES.length); }), ); + it.effect("rejects loss of canonical OpenAI support and skill metadata", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const paths = yield* Path.Path; + const generated = yield* makeGeneratedTargets(pluginRoot); + const openai = generated.targets.openai; + const manifestPath = paths.join(openai, ".codex-plugin", "plugin.json"); + const manifest = yield* fs.readFileString(manifestPath); + yield* fs.writeFileString( + manifestPath, + manifest.replace("https://askgina.ai/support", "https://invalid.example/support"), + ); + yield* fs.remove( + paths.join(openai, "skills", "research-hyperliquid", "agents", "openai.yaml"), + ); + + const report = yield* checkGeneratedTargetConformance("openai", openai, { + packageRoot: pluginRoot, + }); + const failed = report.checks.filter((check) => !check.passed).map((check) => check.id); + assert.include(failed, "openai.manifest.contract"); + assert.include(failed, "openai.skill.research-hyperliquid.openai_metadata"); + }), + ), + ); }); }); diff --git a/plugins/ask-gina/__tests__/sync-plugin-skills.test.ts b/plugins/ask-gina/__tests__/sync-plugin-skills.test.ts index 89e13be..8f58610 100644 --- a/plugins/ask-gina/__tests__/sync-plugin-skills.test.ts +++ b/plugins/ask-gina/__tests__/sync-plugin-skills.test.ts @@ -6,7 +6,7 @@ import { createGeneratedPluginTargets, SKILL_NAMES, TARGET_NAMES, -} from "../../../tools/sync-plugin-skills.js"; +} from "../../../tools/sync-plugin-skills"; const makeGeneratedTargets = Effect.acquireRelease( createGeneratedPluginTargets(), diff --git a/plugins/ask-gina/package.json b/plugins/ask-gina/package.json index 7d166df..031626c 100644 --- a/plugins/ask-gina/package.json +++ b/plugins/ask-gina/package.json @@ -3,7 +3,7 @@ "version": "0.1.0", "license": "Apache-2.0", "files": [ - "src", + "dist", "plugin.yaml", "skills", "evals/model/v1/activation.yaml", @@ -14,8 +14,16 @@ "README.md" ], "type": "module", + "main": "./dist/index.js", + "types": "./dist/index.d.ts", "exports": { - ".": "./src/index.ts" + ".": { + "types": "./dist/index.d.ts", + "import": "./dist/index.js" + } + }, + "scripts": { + "build": "vp -C ../.. pack --filter plugin-core" }, "dependencies": { "@askgina/contracts": "workspace:*", diff --git a/plugins/ask-gina/src/index.ts b/plugins/ask-gina/src/index.ts index 861d007..56962db 100644 --- a/plugins/ask-gina/src/index.ts +++ b/plugins/ask-gina/src/index.ts @@ -7,4 +7,4 @@ export { type AskGinaPluginManifest, type AskGinaPluginSource, type CanonicalSkillDocument, -} from "./loaders.js"; +} from "./loaders"; diff --git a/tools/__tests__/archive-security.test.ts b/tools/__tests__/archive-security.test.ts index ded7aae..392e1a4 100644 --- a/tools/__tests__/archive-security.test.ts +++ b/tools/__tests__/archive-security.test.ts @@ -3,11 +3,7 @@ import { assert, describe, it } from "@effect/vitest"; import { gzipSync } from "node:zlib"; import { Effect, FileSystem, Path } from "effect"; -import { - ArchiveSecurityError, - copyCheckedRegularFile, - preflightTarGz, -} from "../archive-security.js"; +import { ArchiveSecurityError, copyCheckedRegularFile, preflightTarGz } from "../archive-security"; const BLOCK_BYTES = 512; const encoder = new TextEncoder(); @@ -145,14 +141,16 @@ describe("archive security", () => { }), ); - it.effect("rejects a declared file size above the configured resource bound", () => + it.effect("keeps compiled JavaScript, declarations, and maps inside file-size bounds", () => Effect.gen(function* () { - const error = yield* rejectFixture(tarGz({ path: "large", body: "12345" }), { - maxFileBytes: 4, - maxExpandedBytes: 4, - }); - assert.instanceOf(error, ArchiveSecurityError); - assert.include(error.message, "file-size"); + for (const path of ["dist/index.js", "dist/index.d.ts", "dist/index.js.map"]) { + const error = yield* rejectFixture(tarGz({ path, body: "12345" }), { + maxFileBytes: 4, + maxExpandedBytes: 4, + }); + assert.instanceOf(error, ArchiveSecurityError); + assert.include(error.message, "file-size"); + } }), ); }); diff --git a/tools/__tests__/check-typescript-imports.test.ts b/tools/__tests__/check-typescript-imports.test.ts new file mode 100644 index 0000000..7d506fa --- /dev/null +++ b/tools/__tests__/check-typescript-imports.test.ts @@ -0,0 +1,48 @@ +import * as BunServices from "@effect/platform-bun/BunServices"; +import { assert, describe, it } from "@effect/vitest"; +import { Effect, FileSystem, Path } from "effect"; + +import { findTypeScriptJsSpecifiers } from "../check-typescript-imports"; + +describe("TypeScript import specifiers", () => { + it.layer(BunServices.layer)((it) => { + it.effect("rejects only relative .js specifiers that resolve to TypeScript", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "typescript-import-test-" }); + const sourceRoot = path.join(root, "packages", "fixture", "src"); + yield* fs.makeDirectory(sourceRoot, { recursive: true }); + yield* fs.writeFileString( + path.join(sourceRoot, "target.ts"), + "export const value = 1;\n", + ); + yield* fs.writeFileString( + path.join(sourceRoot, "runtime.js"), + "export const value = 1;\n", + ); + yield* fs.writeFileString( + path.join(sourceRoot, "source.ts"), + [ + 'import "./target.js";', + 'import "./runtime.js";', + 'import "./target";', + 'import "@modelcontextprotocol/sdk/client.js";', + 'import "https://example.com/module.js";', + "", + ].join("\n"), + ); + + assert.deepStrictEqual(yield* findTypeScriptJsSpecifiers(root), [ + { + file: "packages/fixture/src/source.ts", + line: 1, + specifier: "./target.js", + }, + ]); + }), + ), + ); + }); +}); diff --git a/tools/__tests__/pack-artifacts.test.ts b/tools/__tests__/pack-artifacts.test.ts index 4404e74..5715043 100644 --- a/tools/__tests__/pack-artifacts.test.ts +++ b/tools/__tests__/pack-artifacts.test.ts @@ -1,14 +1,22 @@ import * as BunServices from "@effect/platform-bun/BunServices"; import { assert, describe, it } from "@effect/vitest"; import { ChildProcess } from "effect/unstable/process"; -import { Data, Effect, FileSystem, Path } from "effect"; +import { Data, Effect, FileSystem, Path, Schema } from "effect"; -import { ArtifactPackError, buildArtifacts } from "../pack-artifacts.js"; +import { + ArtifactPackError, + buildArtifacts, + stagePackage, + verifyCompiledPackageOutput, +} from "../pack-artifacts"; class TestCommandError extends Data.TaggedError("TestCommandError")<{ readonly command: string; readonly exitCode: number; }> {} +const encodeJson = Schema.encodeSync(Schema.fromJsonString(Schema.Unknown)); +const json = (value: unknown): string => `${encodeJson(value)}\n`; + const SOURCE_DIRECTORIES = [ "packages/contracts", "packages/sdk", @@ -71,6 +79,41 @@ const repositoryFixture = Effect.gen(function* () { ); return { root, dist, impact }; }); +const compiledContractFixture = Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const root = yield* fs.makeTempDirectoryScoped({ prefix: "compiled-contract-test-" }); + const packageRoot = path.join(root, "packages/contracts"); + const dist = path.join(packageRoot, "dist"); + const source = "export const value = 1;\n"; + yield* fs.makeDirectory(path.join(packageRoot, "src"), { recursive: true }); + yield* fs.makeDirectory(dist, { recursive: true }); + yield* fs.writeFileString( + path.join(packageRoot, "package.json"), + json({ name: "@askgina/contracts", version: "0.1.0", files: ["dist", "LICENSE", "README.md"] }), + ); + yield* fs.writeFileString(path.join(packageRoot, "LICENSE"), "fixture license\n"); + yield* fs.writeFileString(path.join(packageRoot, "README.md"), "fixture readme\n"); + yield* fs.writeFileString(path.join(packageRoot, "src/index.ts"), source); + yield* fs.writeFileString( + path.join(dist, "index.js"), + "const value = 1;\n//# sourceMappingURL=index.js.map\n", + ); + yield* fs.writeFileString(path.join(dist, "index.d.ts"), "declare const value = 1;\n"); + const writeMap = (value: unknown) => + fs.writeFileString(path.join(dist, "index.js.map"), json(value)); + yield* writeMap({ + version: 3, + sources: ["../src/index.ts"], + sourcesContent: [source], + names: [], + mappings: "", + }); + return { root, packageRoot, dist, source, writeMap }; +}); + +const rejectCompiledContract = (root: string) => + verifyCompiledPackageOutput(root, root, "@askgina/contracts").pipe(Effect.flip); const assertRejectedBeforeImpact = (root: string, dist: string, impact: string) => Effect.gen(function* () { @@ -84,6 +127,147 @@ const assertRejectedBeforeImpact = (root: string, dist: string, impact: string) describe("pack artifact source snapshot", () => { it.layer(BunServices.layer)((it) => { + it.effect("accepts compiled output with matching embedded TypeScript", () => + Effect.scoped( + Effect.gen(function* () { + const path = yield* Path.Path; + const fixture = yield* compiledContractFixture; + const result = yield* verifyCompiledPackageOutput( + fixture.root, + fixture.root, + "@askgina/contracts", + ); + assert.deepStrictEqual(result.files, ["index.d.ts", "index.js", "index.js.map"]); + const staged = yield* stagePackage( + fixture.root, + fixture.root, + "@askgina/contracts", + path.join(fixture.root, "stage"), + "0.1.0", + ); + assert.isTrue(staged.some((proof) => proof.path === "package/dist/index.js")); + }), + ), + ); + + it.effect("rejects modified ignored executable bytes before staging", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const fixture = yield* compiledContractFixture; + const committedRoot = yield* fs.makeTempDirectoryScoped({ + prefix: "committed-contract-test-", + }); + const committedPackageRoot = path.join(committedRoot, "packages/contracts"); + yield* fs.makeDirectory(path.dirname(committedPackageRoot), { recursive: true }); + yield* fs.copy(fixture.packageRoot, committedPackageRoot, { overwrite: true }); + yield* fs.writeFileString( + path.join(fixture.dist, "index.js"), + "globalThis.compromised = true;\n//# sourceMappingURL=index.js.map\n", + ); + const stage = path.join(fixture.root, "stage"); + const error = yield* stagePackage( + fixture.root, + committedRoot, + "@askgina/contracts", + stage, + "0.1.0", + ).pipe(Effect.flip); + assert.instanceOf(error, ArtifactPackError); + assert.include(error.message, "compiled output differs from source commit build"); + assert.isFalse(yield* fs.exists(stage)); + }), + ), + ); + + it.effect("rejects missing and stale embedded source content", () => + Effect.scoped( + Effect.gen(function* () { + const fixture = yield* compiledContractFixture; + yield* fixture.writeMap({ version: 3, sources: ["../src/index.ts"] }); + const missing = yield* rejectCompiledContract(fixture.root); + assert.include(missing.message, "sourcesContent must match sources"); + + yield* fixture.writeMap({ + version: 3, + sources: ["../src/index.ts"], + sourcesContent: ["export const value = 2;\n"], + }); + const stale = yield* rejectCompiledContract(fixture.root); + assert.include(stale.message, "content is stale"); + }), + ), + ); + + it.effect("rejects absolute host paths in source maps", () => + Effect.scoped( + Effect.gen(function* () { + const fixture = yield* compiledContractFixture; + yield* fixture.writeMap({ + version: 3, + sources: ["/home/private/checkout/src/index.ts"], + sourcesContent: [fixture.source], + }); + const error = yield* rejectCompiledContract(fixture.root); + assert.include(error.message, "unsafe source path"); + }), + ), + ); + + it.effect("rejects raw-source allowlists and unexpected compiled files", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const fixture = yield* compiledContractFixture; + yield* fs.writeFileString( + path.join(fixture.packageRoot, "package.json"), + json({ files: ["dist", "src", "LICENSE", "README.md"] }), + ); + const rawSource = yield* rejectCompiledContract(fixture.root); + assert.include(rawSource.message, "package files are inconsistent"); + + yield* fs.writeFileString( + path.join(fixture.packageRoot, "package.json"), + json({ files: ["dist", "LICENSE", "README.md"] }), + ); + yield* fs.writeFileString(path.join(fixture.dist, "unexpected.js"), "export {};\n"); + const unexpected = yield* rejectCompiledContract(fixture.root); + assert.include(unexpected.message, "unexpected compiled output"); + }), + ), + ); + + it.effect("rejects missing compiled files", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const fixture = yield* compiledContractFixture; + yield* fs.remove(path.join(fixture.dist, "index.js")); + const error = yield* rejectCompiledContract(fixture.root); + assert.include(error.message, "compiled output is missing or ambiguous"); + }), + ), + ); + + it.effect("rejects a declaration reference to an omitted map", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const fixture = yield* compiledContractFixture; + yield* fs.writeFileString( + path.join(fixture.dist, "index.d.ts"), + "declare const value = 1;\n//# sourceMappingURL=index.d.ts.map\n", + ); + const error = yield* rejectCompiledContract(fixture.root); + assert.include(error.message, "compiled declaration references a missing source map"); + }), + ), + ); + it.effect("rejects a package wrapper symlink before archive output", () => Effect.scoped( Effect.gen(function* () { diff --git a/tools/__tests__/verify-artifacts.test.ts b/tools/__tests__/verify-artifacts.test.ts index 3918fae..58e0295 100644 --- a/tools/__tests__/verify-artifacts.test.ts +++ b/tools/__tests__/verify-artifacts.test.ts @@ -1,9 +1,15 @@ import * as BunServices from "@effect/platform-bun/BunServices"; import { assert, describe, it } from "@effect/vitest"; import { ChildProcess } from "effect/unstable/process"; -import { Effect, FileSystem, Path } from "effect"; +import { Config, Effect, FileSystem, Path } from "effect"; -import { snapshotArtifactInputs } from "../verify-artifacts.js"; +import { findEmbeddedSourceMapBoundaryRules, inspectSourceMapText } from "../check-public-boundary"; +import { + runNodeEsmSmoke, + snapshotArtifactInputs, + verifyNoInstalledLibrarySources, + verifyNode24Consumer, +} from "../verify-artifacts"; describe("artifact verification snapshots", () => { it.layer(BunServices.layer)((it) => { @@ -47,5 +53,122 @@ describe("artifact verification snapshots", () => { }), ), ); + it.effect("fails closed when Node 24 is unavailable", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const directory = yield* fs.makeTempDirectoryScoped({ prefix: "node24-missing-test-" }); + const missingNode = path.join(directory, "missing-node"); + const error = yield* verifyNode24Consumer({ + node: missingNode, + project: directory, + env: { PATH: "/usr/bin:/bin" }, + }).pipe(Effect.flip); + assert.include(error.message, `cannot start ${missingNode}`); + }), + ), + ); + + it.effect("rejects raw installed library sources", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const directory = yield* fs.makeTempDirectoryScoped({ prefix: "raw-source-test-" }); + yield* fs.makeDirectory(path.join(directory, "node_modules/@askgina/sdk/src"), { + recursive: true, + }); + const error = yield* verifyNoInstalledLibrarySources(directory).pipe(Effect.flip); + assert.strictEqual(error.message, "@askgina/sdk installed raw source"); + }), + ), + ); + + it.effect("rejects broken exports and undeclared runtime imports", () => + Effect.scoped( + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const directory = yield* fs.makeTempDirectoryScoped({ prefix: "node-esm-test-" }); + const packageRoot = path.join(directory, "node_modules/@askgina/sdk"); + const dist = path.join(packageRoot, "dist"); + yield* fs.makeDirectory(dist, { recursive: true }); + const writeManifest = (entry: string) => + fs.writeFileString( + path.join(packageRoot, "package.json"), + `${JSON.stringify({ + name: "@askgina/sdk", + type: "module", + exports: { ".": { import: entry } }, + })}\n`, + ); + const env = { + PATH: yield* Config.string("PATH").pipe(Config.withDefault("/usr/bin:/bin")), + HOME: directory, + }; + const importScript = + '// Dynamic import intentionally exercises the installed package export boundary.\nawait import("@askgina/sdk");'; + + yield* writeManifest("./dist/missing.js"); + const brokenExports = yield* runNodeEsmSmoke({ + node: "node", + cwd: directory, + source: importScript, + env, + }).pipe(Effect.flip); + assert.strictEqual(brokenExports.message, "Node ESM smoke failed"); + + yield* writeManifest("./dist/index.js"); + yield* fs.writeFileString( + path.join(dist, "index.js"), + 'import "undeclared-runtime"; export const value = 1;\n', + ); + const undeclaredDependency = yield* runNodeEsmSmoke({ + node: "node", + cwd: directory, + source: importScript, + env, + }).pipe(Effect.flip); + assert.strictEqual(undeclaredDependency.message, "Node ESM smoke failed"); + }), + ), + ); + + it("inspects embedded source without treating map identifiers as imports", () => { + const harmlessText = JSON.stringify({ + version: 3, + names: ["@effect/platform-node"], + sources: ["../src/index.ts"], + sourcesContent: ['const platformNodeIdentifier = "@effect/platform-node";\n'], + }); + const harmless = inspectSourceMapText(harmlessText); + assert.isDefined(harmless); + assert.isFalse(harmless.unsafeSourcePath); + assert.notInclude(findEmbeddedSourceMapBoundaryRules(harmlessText), "forbidden-runtime"); + + const forbiddenRuntime = ["@effect/platform", "-node"].join(""); + const forbiddenText = JSON.stringify({ + version: 3, + sources: ["../src/index.ts"], + sourcesContent: [`import "${forbiddenRuntime}";\n`], + }); + assert.isDefined(inspectSourceMapText(forbiddenText)); + assert.include(findEmbeddedSourceMapBoundaryRules(forbiddenText), "forbidden-runtime"); + assert.isUndefined( + inspectSourceMapText( + JSON.stringify({ version: 3, sources: ["../src/index.ts"], sourcesContent: [] }), + ), + ); + assert.isTrue( + inspectSourceMapText( + JSON.stringify({ + version: 3, + sources: ["/home/private/src/index.ts"], + sourcesContent: ["export {};\n"], + }), + )?.unsafeSourcePath, + ); + }); }); }); diff --git a/tools/build-contract.ts b/tools/build-contract.ts index 12cb5d1..c88bfcc 100755 --- a/tools/build-contract.ts +++ b/tools/build-contract.ts @@ -6,6 +6,11 @@ import { ChildProcess } from "effect/unstable/process"; import { Crypto, Data, Effect, FileSystem, Layer, Path, Schema, Stream } from "effect"; const CONTRACT_SOURCE = "packages/contracts/src/index.ts"; +const CONTRACT_OUTPUTS = [ + "packages/contracts/dist/index.d.ts", + "packages/contracts/dist/index.js", + "packages/contracts/dist/index.js.map", +] as const; const RECEIPT_PATH = "dist/receipts/contract.json"; const SHA_256 = /^[a-f0-9]{64}$/u; const GIT_COMMIT = /^[a-f0-9]{40}$/u; @@ -98,17 +103,45 @@ export const buildContractReceipt = (root: string) => if (catalogSha === undefined || !SHA_256.test(catalogSha)) { return yield* fail("contracts catalogSha must be a SHA-256 digest"); } - const sourceHash = toHex( - yield* crypto - .digest("SHA-256", new TextEncoder().encode(source)) - .pipe(Effect.mapError((cause) => fail("cannot hash public contract source", cause))), + const outputDirectory = path.join(root, "packages/contracts/dist"); + const outputNames = (yield* fs + .readDirectory(outputDirectory) + .pipe( + Effect.mapError((cause) => fail("cannot list compiled contract output", cause)), + )).sort(); + if (outputNames.join("\n") !== CONTRACT_OUTPUTS.map((file) => path.basename(file)).join("\n")) { + return yield* fail("compiled contract output is missing or unexpected"); + } + const files = yield* Effect.forEach(CONTRACT_OUTPUTS, (file) => + Effect.gen(function* () { + const absolute = path.join(root, file); + const symbolicLink = yield* fs + .readLink(absolute) + .pipe(Effect.match({ onFailure: () => false, onSuccess: () => true })); + if (symbolicLink) + return yield* fail(`compiled contract output is a symbolic link: ${file}`); + const info = yield* fs + .stat(absolute) + .pipe(Effect.mapError((cause) => fail(`cannot inspect ${file}`, cause))); + if (info.type !== "File") + return yield* fail(`compiled contract output is not a file: ${file}`); + const bytes = yield* fs + .readFile(absolute) + .pipe(Effect.mapError((cause) => fail(`cannot read ${file}`, cause))); + const sha256 = toHex( + yield* crypto + .digest("SHA-256", bytes) + .pipe(Effect.mapError((cause) => fail(`cannot hash ${file}`, cause))), + ); + return { path: file, sha256 }; + }), ); return { schemaVersion: "v1", releaseVersion, sourceCommit: yield* sourceCommit(root), catalogSha, - files: [{ path: CONTRACT_SOURCE, sha256: sourceHash }], + files: files, } satisfies ContractReceipt; }); diff --git a/tools/check-public-boundary.ts b/tools/check-public-boundary.ts index edbbd8f..185f1c9 100755 --- a/tools/check-public-boundary.ts +++ b/tools/check-public-boundary.ts @@ -5,10 +5,10 @@ import * as BunServices from "@effect/platform-bun/BunServices"; import { findPublicTextViolations, type PublicTextViolationKind, -} from "../packages/evals/src/index.js"; +} from "../packages/evals/src/index"; import { Data, Effect, FileSystem, Layer, Path, Schema } from "effect"; -import { extractCheckedTarGz } from "./archive-security.js"; +import { extractCheckedTarGz } from "./archive-security"; const HOSTS = ["openai", "cursor", "claude", "copilot", "gemini"]; const PACKAGES = [ @@ -76,6 +76,97 @@ const fail = (message: string, cause?: unknown) => new PublicBoundaryError(cause === undefined ? { message } : { message, cause }); const isObject = (value: unknown): value is Record => typeof value === "object" && value !== null && !Array.isArray(value); +const ABSOLUTE_OR_URI_SOURCE = /^(?:\/|[A-Za-z]:[\\/]|\\\\|[A-Za-z][A-Za-z\d+.-]*:)/u; + +export const inspectSourceMapText = ( + text: string, +): + | Readonly<{ + readonly sources: readonly string[]; + readonly sourcesContent: readonly string[]; + readonly unsafeSourcePath: boolean; + }> + | undefined => { + let value: unknown; + try { + value = JSON.parse(text); + } catch { + return undefined; + } + if ( + !isObject(value) || + value.version !== 3 || + !Array.isArray(value.sources) || + !Array.isArray(value.sourcesContent) || + value.sources.length !== value.sourcesContent.length || + !value.sources.every((source) => typeof source === "string") || + !value.sourcesContent.every((source) => typeof source === "string") + ) { + return undefined; + } + return { + sources: value.sources, + sourcesContent: value.sourcesContent, + unsafeSourcePath: value.sources.some((source) => ABSOLUTE_OR_URI_SOURCE.test(source)), + }; +}; +const findPublicBoundaryTextRules = ( + text: string, + label: string, + receipt: boolean, +): readonly string[] => { + const rules: string[] = []; + const add = (rule: string): void => { + rules.push(rule); + }; + const codeOrData = /\.(?:[cm]?[jt]sx?|json|ya?ml|toml|lock)$/iu.test(label); + if (text.includes(PRIVATE_REPOSITORY)) add("private-repository-name"); + if (codeOrData && text.includes(PRIVATE_REGISTRY)) add("private-registry"); + if (codeOrData && text.includes(PRIVATE_REGISTRY_FIELD)) add("private-registry-field"); + for (const match of text.matchAll( + /(?:from\s+|import\s*(?:\(\s*)?|require\s*\()\s*["']([^"']+)["']/gu, + )) { + const specifier = match[1] ?? ""; + if (specifier.startsWith(PRIVATE_ALIAS)) add("private-import"); + if (specifier === FORBIDDEN_RUNTIME || specifier.startsWith(`${FORBIDDEN_RUNTIME}/`)) + add("forbidden-runtime"); + if (specifier.toLowerCase().includes(PRIVATE_CACHE)) add("private-cache"); + } + if (/(?:package\.json|bun\.lock)(?::|$)/u.test(label)) { + for (const match of text.matchAll(/["']([^"']+)["']\s*:/gu)) { + const dependency = (match[1] ?? "").toLowerCase(); + if (dependency === FORBIDDEN_RUNTIME || dependency.startsWith(`${FORBIDDEN_RUNTIME}/`)) + add("forbidden-runtime"); + if (dependency.includes(PRIVATE_CACHE)) add("private-cache"); + } + } + if (/\bREDIS_[A-Z0-9_]+\b|rediss?:\/\//u.test(text)) add("private-cache"); + for (const violation of reportablePublicTextViolations(text, receipt)) add(violation.kind); + for (const match of text.matchAll(/https?:\/\/([^\s/"'<>]+)/giu)) { + const hostname = (match[1] ?? "").toLowerCase().replace(/:\d+$/u, ""); + if ( + hostname === "localhost" || + hostname.endsWith(".local") || + hostname.endsWith(".internal") || + hostname.endsWith(".private") || + /^(?:127\.|0\.0\.0\.0$|10\.|192\.168\.|172\.(?:1[6-9]|2\d|3[01])\.)/u.test(hostname) + ) + add("private-host"); + } + if (receipt && RAW_EVAL_FIELDS.test(text)) add("raw-eval-field"); + return rules; +}; +export const findEmbeddedSourceMapBoundaryRules = (text: string): readonly string[] => { + const sourceMap = inspectSourceMapText(text); + if (sourceMap === undefined) return ["invalid-source-map"]; + const rules = sourceMap.unsafeSourcePath ? ["absolute-source-map-path"] : []; + for (const [index, source] of sourceMap.sourcesContent.entries()) { + rules.push( + ...findPublicBoundaryTextRules(source, sourceMap.sources[index] ?? String(index), false), + ); + } + return rules; +}; const readText = (file: string) => Effect.gen(function* () { @@ -173,44 +264,9 @@ const program = Effect.scoped( findings.push({ rule, path: file.split(path.sep).join("/") }); }; const scanText = (text: string, label: string, receipt: boolean): void => { - const codeOrData = /\.(?:[cm]?[jt]sx?|json|ya?ml|toml|lock)$/iu.test(label); - if (text.includes(PRIVATE_REPOSITORY)) addFinding("private-repository-name", label); - if (codeOrData && text.includes(PRIVATE_REGISTRY)) addFinding("private-registry", label); - if (codeOrData && text.includes(PRIVATE_REGISTRY_FIELD)) - addFinding("private-registry-field", label); - for (const match of text.matchAll( - /(?:from\s+|import\s*(?:\(\s*)?|require\s*\()\s*["']([^"']+)["']/gu, - )) { - const specifier = match[1] ?? ""; - if (specifier.startsWith(PRIVATE_ALIAS)) addFinding("private-import", label); - if (specifier === FORBIDDEN_RUNTIME || specifier.startsWith(`${FORBIDDEN_RUNTIME}/`)) - addFinding("forbidden-runtime", label); - if (specifier.toLowerCase().includes(PRIVATE_CACHE)) addFinding("private-cache", label); + for (const rule of findPublicBoundaryTextRules(text, label, receipt)) { + addFinding(rule, label); } - if (/(?:package\.json|bun\.lock)(?::|$)/u.test(label)) { - for (const match of text.matchAll(/["']([^"']+)["']\s*:/gu)) { - const dependency = (match[1] ?? "").toLowerCase(); - if (dependency === FORBIDDEN_RUNTIME || dependency.startsWith(`${FORBIDDEN_RUNTIME}/`)) - addFinding("forbidden-runtime", label); - if (dependency.includes(PRIVATE_CACHE)) addFinding("private-cache", label); - } - } - if (/\bREDIS_[A-Z0-9_]+\b|rediss?:\/\//u.test(text)) addFinding("private-cache", label); - for (const violation of reportablePublicTextViolations(text, receipt)) { - addFinding(violation.kind, label); - } - for (const match of text.matchAll(/https?:\/\/([^\s/"'<>]+)/giu)) { - const hostname = (match[1] ?? "").toLowerCase().replace(/:\d+$/u, ""); - if ( - hostname === "localhost" || - hostname.endsWith(".local") || - hostname.endsWith(".internal") || - hostname.endsWith(".private") || - /^(?:127\.|0\.0\.0\.0$|10\.|192\.168\.|172\.(?:1[6-9]|2\d|3[01])\.)/u.test(hostname) - ) - addFinding("private-host", label); - } - if (receipt && RAW_EVAL_FIELDS.test(text)) addFinding("raw-eval-field", label); }; const scanFile = (absolute: string, label: string, receipt: boolean) => Effect.gen(function* () { @@ -226,7 +282,20 @@ const program = Effect.scoped( .readFile(absolute) .pipe(Effect.mapError((cause) => fail(`cannot read ${absolute}`, cause))); if (bytes.includes(0)) addFinding("unscannable-binary-file", label); - else scanText(new TextDecoder().decode(bytes), label, receipt); + else { + const text = new TextDecoder().decode(bytes); + scanText(text, label, receipt); + if (label.endsWith(".map")) { + const sourceMap = inspectSourceMapText(text); + if (sourceMap === undefined) addFinding("invalid-source-map", label); + else { + if (sourceMap.unsafeSourcePath) addFinding("absolute-source-map-path", label); + for (const [index, source] of sourceMap.sourcesContent.entries()) { + scanText(source, `${label}#${sourceMap.sources[index] ?? index}`, false); + } + } + } + } }); const comparePackageDeclarations = (stage: string, definition: PackageDefinition) => Effect.gen(function* () { @@ -336,6 +405,11 @@ const program = Effect.scoped( (item) => relative === `packages/askgina-${item.slug}-${version}.tgz`, ); if (definition !== undefined) { + for (const file of tree.files) { + if (/\.(?:[cm]?ts|tsx)$/u.test(file) && !/\.d\.(?:[cm]?ts|tsx)$/u.test(file)) { + addFinding("raw-package-source", `dist/${relative}:${file}`); + } + } yield* comparePackageDeclarations(stage, definition).pipe( Effect.catchIf( () => true, diff --git a/tools/check-target-conformance.ts b/tools/check-target-conformance.ts index 13f0dca..5aaa71c 100755 --- a/tools/check-target-conformance.ts +++ b/tools/check-target-conformance.ts @@ -31,7 +31,7 @@ import { createGeneratedPluginTarget, type PluginSkillSyncError, type TargetName, -} from "./sync-plugin-skills.js"; +} from "./sync-plugin-skills"; export { TARGET_NAMES, type TargetName }; diff --git a/tools/check-typescript-imports.ts b/tools/check-typescript-imports.ts new file mode 100755 index 0000000..5fae840 --- /dev/null +++ b/tools/check-typescript-imports.ts @@ -0,0 +1,123 @@ +#!/usr/bin/env bun + +import * as BunRuntime from "@effect/platform-bun/BunRuntime"; +import * as BunServices from "@effect/platform-bun/BunServices"; +import { Console, Data, Effect, FileSystem, Layer, Path } from "effect"; + +export interface TypeScriptJsSpecifierFinding { + readonly file: string; + readonly line: number; + readonly specifier: string; +} + +export class TypeScriptImportCheckError extends Data.TaggedError("TypeScriptImportCheckError")<{ + readonly message: string; + readonly cause?: unknown; +}> {} +const fail = (message: string, cause?: unknown): TypeScriptImportCheckError => + new TypeScriptImportCheckError(cause === undefined ? { message } : { message, cause }); + +const listTypeScriptFiles = (root: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const visit = ( + directory: string, + prefix: string, + ): Effect.Effect => + Effect.gen(function* () { + const names = yield* fs + .readDirectory(directory) + .pipe(Effect.mapError((cause) => fail(`cannot list ${directory}`, cause))); + const nested = yield* Effect.forEach(names.sort(), (name) => + Effect.gen(function* () { + if (name === "node_modules" || name === "dist") return []; + const absolute = path.join(directory, name); + const relative = prefix.length > 0 ? `${prefix}/${name}` : name; + const info = yield* fs + .stat(absolute) + .pipe(Effect.mapError((cause) => fail(`cannot inspect ${absolute}`, cause))); + if (info.type === "Directory") return yield* visit(absolute, relative); + return info.type === "File" && name.endsWith(".ts") ? [relative] : []; + }), + ); + return nested.flat(); + }); + + const files = (yield* fs.exists(path.join(root, "vite.config.ts"))) ? ["vite.config.ts"] : []; + for (const sourceRoot of ["packages", "plugins", "scripts", "tools"]) { + const directory = path.join(root, sourceRoot); + if (!(yield* fs.exists(directory))) continue; + const nested = yield* visit(directory, sourceRoot); + files.push(...nested); + } + return files.sort(); + }); + +export const findTypeScriptJsSpecifiers = (root: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const files = yield* listTypeScriptFiles(root); + const findings: TypeScriptJsSpecifierFinding[] = []; + + for (const relativeFile of files) { + const file = path.join(root, relativeFile); + if (!(yield* fs.exists(file))) continue; + const source = yield* fs + .readFileString(file) + .pipe(Effect.mapError((cause) => fail(`cannot read ${relativeFile}`, cause))); + const specifiers = source.matchAll(/(["'])((?:\.\.?\/)[^"'\n]+\.js)\1/gu); + for (const match of specifiers) { + const specifier = match[2]; + if (specifier === undefined || match.index === undefined) continue; + const base = path.resolve(path.dirname(file), specifier.slice(0, -3)); + const candidates = [ + `${base}.ts`, + `${base}.tsx`, + path.join(base, "index.ts"), + path.join(base, "index.tsx"), + ]; + let resolvesToTypeScript = false; + for (const candidate of candidates) { + if (yield* fs.exists(candidate)) { + resolvesToTypeScript = true; + break; + } + } + if (!resolvesToTypeScript) continue; + findings.push({ + file: relativeFile, + line: source.slice(0, match.index).split("\n").length, + specifier, + }); + } + } + + return findings; + }); + +export const checkTypeScriptImports = (root: string) => + findTypeScriptJsSpecifiers(root).pipe( + Effect.flatMap((findings) => + findings.length === 0 + ? Console.log("TypeScript import specifiers are extensionless") + : Effect.fail( + fail( + findings + .map(({ file, line, specifier }) => `${file}:${line}: ${specifier}`) + .join("\n"), + ), + ), + ), + ); + +if (import.meta.main) { + const main = Layer.build(BunServices.layer).pipe( + Effect.flatMap((context) => + checkTypeScriptImports(process.cwd()).pipe(Effect.provide(context)), + ), + Effect.scoped, + ); + BunRuntime.runMain(main); +} diff --git a/tools/pack-artifacts.ts b/tools/pack-artifacts.ts index 0259d7d..6cd7de8 100755 --- a/tools/pack-artifacts.ts +++ b/tools/pack-artifacts.ts @@ -3,11 +3,11 @@ import * as BunRuntime from "@effect/platform-bun/BunRuntime"; import * as BunServices from "@effect/platform-bun/BunServices"; import { ChildProcess } from "effect/unstable/process"; -import { Crypto, Data, Effect, FileSystem, Layer, Path, Schema, Stream } from "effect"; +import { Crypto, Data, Effect, FileSystem, Function, Layer, Path, Schema, Stream } from "effect"; -import { runHermeticEvalReplay, sanitizeEvalReplay } from "../packages/evals/src/index.js"; -import { copyCheckedRegularFile } from "./archive-security.js"; -import { checkGeneratedTargetConformance } from "./check-target-conformance.js"; +import { runHermeticEvalReplay, sanitizeEvalReplay } from "../packages/evals/src/index"; +import { copyCheckedRegularFile } from "./archive-security"; +import { checkGeneratedTargetConformance } from "./check-target-conformance"; const HOSTS = ["openai", "cursor", "claude", "copilot", "gemini"] as const; const SKILLS = [ @@ -17,11 +17,74 @@ const SKILLS = [ "review-gina-account", ]; const PACKAGES = [ - { slug: "contracts", name: "@askgina/contracts", directory: "packages/contracts" }, - { slug: "sdk", name: "@askgina/sdk", directory: "packages/sdk" }, - { slug: "cli", name: "@askgina/cli", directory: "packages/cli" }, - { slug: "plugin-core", name: "@askgina/plugin-core", directory: "plugins/ask-gina" }, - { slug: "evals", name: "@askgina/evals", directory: "packages/evals" }, + { + slug: "contracts", + name: "@askgina/contracts", + directory: "packages/contracts", + packageFiles: ["dist", "LICENSE", "README.md"], + compiledFiles: [/^index\.d\.ts$/u, /^index\.js$/u, /^index\.js\.map$/u], + }, + { + slug: "sdk", + name: "@askgina/sdk", + directory: "packages/sdk", + packageFiles: ["dist", "LICENSE", "README.md"], + compiledFiles: [/^index\.d\.ts$/u, /^index\.js$/u, /^index\.js\.map$/u], + }, + { + slug: "cli", + name: "@askgina/cli", + directory: "packages/cli", + packageFiles: ["dist", "LICENSE", "README.md"], + compiledFiles: [ + /^bin\.d\.ts$/u, + /^bin\.js$/u, + /^bin\.js\.map$/u, + /^index\.d\.ts$/u, + /^index\.js$/u, + /^run-[A-Za-z0-9_-]+\.js$/u, + /^run-[A-Za-z0-9_-]+\.js\.map$/u, + ], + }, + { + slug: "plugin-core", + name: "@askgina/plugin-core", + directory: "plugins/ask-gina", + packageFiles: [ + "dist", + "plugin.yaml", + "skills", + "evals/model/v1/activation.yaml", + "evals/model/v1/smoke.yaml", + "evals/model/v1/families", + "evals/model/v1/fixtures", + "LICENSE", + "README.md", + ], + compiledFiles: [/^index\.d\.ts$/u, /^index\.js$/u, /^index\.js\.map$/u], + }, + { + slug: "evals", + name: "@askgina/evals", + directory: "packages/evals", + packageFiles: ["dist", "LICENSE", "README.md"], + compiledFiles: [ + /^bin\/live\.d\.ts$/u, + /^bin\/live\.js$/u, + /^bin\/live\.js\.map$/u, + /^bin\/replay\.d\.ts$/u, + /^bin\/replay\.js$/u, + /^bin\/replay\.js\.map$/u, + /^index\.d\.ts$/u, + /^index\.js$/u, + /^report-[A-Za-z0-9_-]+\.js$/u, + /^report-[A-Za-z0-9_-]+\.js\.map$/u, + /^responses-api-[A-Za-z0-9_-]+\.js$/u, + /^responses-api-[A-Za-z0-9_-]+\.js\.map$/u, + /^runner-[A-Za-z0-9_-]+\.js$/u, + /^runner-[A-Za-z0-9_-]+\.js\.map$/u, + ], + }, ]; const TARGET_MANIFESTS: Readonly> = { openai: ".codex-plugin/plugin.json", @@ -38,8 +101,6 @@ const MAX_GIT_PORCELAIN_BYTES = 64 * 1024; const RAW_EVAL_FIELDS = /"(?:prompts?|toolCalls?|tool_calls|payloads?|models?|accounts?|addresses?|final_answer|report)"\s*:/iu; -type PackageDefinition = (typeof PACKAGES)[number]; - export class ArtifactPackError extends Data.TaggedError("ArtifactPackError")<{ readonly message: string; readonly cause?: unknown; @@ -191,12 +252,17 @@ const commandHasBoundedOutput = ( }), ); -const runCommand = (command: string, args: readonly string[], cwd: string) => +const runCommand = ( + command: string, + args: readonly string[], + cwd: string, + environment: Readonly> = {}, +) => Effect.scoped( Effect.gen(function* () { const child = yield* ChildProcess.make(command, args, { cwd, - env: childEnvironment(), + env: { ...childEnvironment(), ...environment }, extendEnv: false, stdin: "ignore", stdout: "ignore", @@ -251,6 +317,186 @@ const fileProofs = (directory: string, prefix = "") => ), ); }); +const packageDefinition = (name: string) => { + const definition = PACKAGES.find((candidate) => candidate.name === name); + return definition === undefined + ? Effect.fail(fail(`unknown package definition: ${name}`)) + : Effect.succeed(definition); +}; + +const verifyEmbeddedSourceMap = ( + livePackageRoot: string, + sourcePackageRoot: string, + mapFile: string, +) => + Effect.gen(function* () { + const path = yield* Path.Path; + const value = yield* readJson(mapFile); + if (!isObject(value) || value.version !== 3) { + return yield* fail("compiled source map must be a version 3 object"); + } + if (value.sourceRoot !== undefined && value.sourceRoot !== "") { + return yield* fail("compiled source map must not declare sourceRoot"); + } + const companion = mapFile.slice(0, -".map".length); + const companionName = path.basename(companion); + if (value.file !== undefined && value.file !== companionName) { + return yield* fail("compiled source map file does not match its companion"); + } + const compiled = yield* readText(companion); + if (!compiled.trimEnd().endsWith(`//# sourceMappingURL=${path.basename(mapFile)}`)) { + return yield* fail("compiled file does not reference its source map"); + } + if (!Array.isArray(value.sources) || value.sources.length === 0) { + return yield* fail("compiled source map must contain sources"); + } + if ( + !Array.isArray(value.sourcesContent) || + value.sourcesContent.length !== value.sources.length + ) { + return yield* fail("compiled source map sourcesContent must match sources"); + } + for (const [index, sourceValue] of value.sources.entries()) { + const content = value.sourcesContent[index]; + if (typeof sourceValue !== "string" || sourceValue.length === 0) { + return yield* fail("compiled source map contains an invalid source path"); + } + if (typeof content !== "string" || content.length === 0) { + return yield* fail("compiled source map contains empty sourcesContent"); + } + if ( + sourceValue.includes("\\") || + sourceValue.startsWith("/") || + /^[A-Za-z]:/u.test(sourceValue) || + /^[A-Za-z][A-Za-z0-9+.-]*:/u.test(sourceValue) || + Array.from(sourceValue).some((character) => { + const code = character.charCodeAt(0); + return code <= 0x1f || code === 0x7f; + }) || + sourceValue.split("/").some((segment) => segment.length === 0 || segment === ".") + ) { + return yield* fail(`compiled source map contains an unsafe source path: ${sourceValue}`); + } + const liveSource = path.resolve(path.dirname(mapFile), sourceValue); + const packageRelative = path.relative(livePackageRoot, liveSource); + if ( + packageRelative === ".." || + packageRelative.startsWith("../") || + packageRelative.startsWith("..\\") || + path.isAbsolute(packageRelative) || + !packageRelative.endsWith(".ts") + ) { + return yield* fail(`compiled source map escapes its package: ${sourceValue}`); + } + const expected = yield* readText(path.join(sourcePackageRoot, packageRelative)); + if (content !== expected) { + return yield* fail(`compiled source map content is stale: ${sourceValue}`); + } + } + }); + +export interface VerifiedCompiledPackageOutput { + readonly allowlist: readonly string[]; + readonly files: readonly string[]; +} + +export type VerifyCompiledPackageOutputEffect = Effect.Effect< + VerifiedCompiledPackageOutput, + ArtifactPackError, + FileSystem.FileSystem | Path.Path +>; + +const verifyCompiledPackageOutputImpl = ( + liveRoot: string, + sourceRoot: string, + packageName: string, +): VerifyCompiledPackageOutputEffect => + Effect.gen(function* () { + const definition = yield* packageDefinition(packageName); + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const livePackageRoot = path.join(liveRoot, definition.directory); + const sourcePackageRoot = path.join(sourceRoot, definition.directory); + const metadata = yield* readJson(path.join(sourcePackageRoot, "package.json")); + if (!isObject(metadata)) { + return yield* fail(`${definition.name} package.json must be an object`); + } + const allowlist = yield* normalizedPackageFiles(metadata.files, `${definition.name} files`); + if (stableJson(allowlist) !== stableJson(definition.packageFiles)) { + return yield* fail(`${definition.name} package files are inconsistent`); + } + const dist = path.join(livePackageRoot, "dist"); + const files = yield* filesBelow(dist); + const committedDist = path.join(sourcePackageRoot, "dist"); + const committedFiles = yield* filesBelow(committedDist); + if (files.join("\n") !== committedFiles.join("\n")) { + return yield* fail(`${definition.name} compiled output differs from source commit build`); + } + yield* Effect.forEach(files, (file) => + Effect.gen(function* () { + const liveBytes = yield* fs + .readFile(path.join(dist, file)) + .pipe(Effect.mapError((cause) => fail(`cannot read ${definition.name}:${file}`, cause))); + const committedBytes = yield* fs + .readFile(path.join(committedDist, file)) + .pipe( + Effect.mapError((cause) => + fail(`cannot read source commit build ${definition.name}:${file}`, cause), + ), + ); + if ( + liveBytes.length !== committedBytes.length || + liveBytes.some((byte, index) => byte !== committedBytes[index]) + ) { + return yield* fail( + `${definition.name} compiled output differs from source commit build: ${file}`, + ); + } + }), + ); + for (const pattern of definition.compiledFiles) { + if (files.filter((file) => pattern.test(file)).length !== 1) { + return yield* fail( + `${definition.name} compiled output is missing or ambiguous: ${pattern}`, + ); + } + } + for (const file of files) { + if (definition.compiledFiles.filter((pattern) => pattern.test(file)).length !== 1) { + return yield* fail(`${definition.name} has unexpected compiled output: ${file}`); + } + } + yield* Effect.forEach( + files.filter((file) => file.endsWith(".d.ts")), + (file) => + Effect.gen(function* () { + const declaration = yield* readText(path.join(dist, file)); + const referencedMap = declaration.match(/\/\/# sourceMappingURL=([^\r\n]+)\s*$/u)?.[1]; + if ( + referencedMap !== undefined && + (referencedMap !== `${path.basename(file)}.map` || !files.includes(`${file}.map`)) + ) { + return yield* fail( + `${definition.name} compiled declaration references a missing source map: ${file}`, + ); + } + }), + ); + const maps = files.filter((file) => file.endsWith(".map")); + if (maps.length === 0) return yield* fail(`${definition.name} has no compiled source maps`); + yield* Effect.forEach(maps, (file) => + verifyEmbeddedSourceMap(livePackageRoot, sourcePackageRoot, path.join(dist, file)), + ); + return { allowlist, files }; + }); + +export const verifyCompiledPackageOutput: { + ( + sourceRoot: string, + packageName: string, + ): (liveRoot: string) => VerifyCompiledPackageOutputEffect; + (liveRoot: string, sourceRoot: string, packageName: string): VerifyCompiledPackageOutputEffect; +} = Function.dual(3, verifyCompiledPackageOutputImpl); const rewriteWorkspaceRanges = (value: unknown, version: string): void => { if (!isObject(value)) return; @@ -263,12 +509,20 @@ const rewriteWorkspaceRanges = (value: unknown, version: string): void => { } }; -const copyPackage = (root: string, definition: PackageDefinition, stage: string, version: string) => +const stagePackageImpl = ( + liveRoot: string, + sourceRoot: string, + packageName: string, + stage: string, + version: string, +) => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; const path = yield* Path.Path; - const sourceRoot = path.join(root, definition.directory); - const metadata = yield* readJson(path.join(sourceRoot, "package.json")); + const definition = yield* packageDefinition(packageName); + const sourcePackageRoot = path.join(sourceRoot, definition.directory); + const verified = yield* verifyCompiledPackageOutput(liveRoot, sourceRoot, definition.name); + const metadata = yield* readJson(path.join(sourcePackageRoot, "package.json")); if (!isObject(metadata)) return yield* fail(`${definition.name} package.json must be an object`); const name = yield* requiredString(metadata.name, `${definition.name} package name`); @@ -279,7 +533,7 @@ const copyPackage = (root: string, definition: PackageDefinition, stage: string, if (name !== definition.name || declaredVersion !== version) { return yield* fail(`${definition.name} package identity is inconsistent`); } - const allowlist = yield* normalizedPackageFiles(metadata.files, `${definition.name} files`); + const allowlist = verified.allowlist; rewriteWorkspaceRanges(metadata, version); const packageRoot = path.join(stage, "package"); yield* fs @@ -290,7 +544,7 @@ const copyPackage = (root: string, definition: PackageDefinition, stage: string, .pipe(Effect.mapError((cause) => fail(`cannot stage ${definition.name} metadata`, cause))); yield* Effect.forEach(allowlist, (entry) => Effect.gen(function* () { - const source = path.join(sourceRoot, entry); + const source = path.join(sourcePackageRoot, entry); const symbolicLink = yield* fs .readLink(source) .pipe(Effect.match({ onFailure: () => false, onSuccess: () => true })); @@ -317,6 +571,21 @@ const copyPackage = (root: string, definition: PackageDefinition, stage: string, ); return yield* fileProofs(packageRoot, "package"); }); +export const stagePackage: { + ( + sourceRoot: string, + packageName: string, + stage: string, + version: string, + ): (liveRoot: string) => ReturnType; + ( + liveRoot: string, + sourceRoot: string, + packageName: string, + stage: string, + version: string, + ): ReturnType; +} = Function.dual(5, stagePackageImpl); const archive = (root: string, stage: string, output: string, entries: readonly string[]) => runCommand( @@ -357,6 +626,90 @@ const snapshotAtCommit = (root: string, sourceCommit: string) => return snapshot; }); +const buildSnapshotPackages = (root: string, snapshot: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const node = Bun.which("node"); + if (node === null) return yield* fail("Node 24 is unavailable for the source commit build"); + const nodeVersion = (yield* commandOutput(node, ["--version"], root)).trim(); + if (!/^v24\./u.test(nodeVersion)) { + return yield* fail(`source commit build requires Node 24, received ${nodeVersion}`); + } + const linkNodeModules = (relative: string, required: boolean) => + Effect.gen(function* () { + const installed = path.join(root, relative, "node_modules"); + if (!(yield* fs.exists(installed))) { + if (required) return yield* fail("installed dependencies are unavailable"); + return; + } + const linked = path.join(snapshot, relative, "node_modules"); + yield* fs + .makeDirectory(linked, { recursive: true }) + .pipe(Effect.mapError((cause) => fail(`cannot create ${relative}/node_modules`, cause))); + const names = yield* fs + .readDirectory(installed) + .pipe(Effect.mapError((cause) => fail(`cannot list ${relative}/node_modules`, cause))); + yield* Effect.forEach(names.sort(), (name) => + Effect.gen(function* () { + if (name === ".vite" || name === ".vite-temp") return; + if (name !== "@askgina") { + yield* fs + .symlink(path.join(installed, name), path.join(linked, name)) + .pipe( + Effect.mapError((cause) => + fail(`cannot link build dependency ${relative}/node_modules/${name}`, cause), + ), + ); + return; + } + const installedScope = path.join(installed, name); + const linkedScope = path.join(linked, name); + yield* fs + .makeDirectory(linkedScope, { recursive: true }) + .pipe(Effect.mapError((cause) => fail("cannot link workspace dependencies", cause))); + const workspaceNames = yield* fs + .readDirectory(installedScope) + .pipe(Effect.mapError((cause) => fail("cannot list workspace dependencies", cause))); + yield* Effect.forEach(workspaceNames.sort(), (workspaceName) => + Effect.gen(function* () { + const definition = PACKAGES.find( + (candidate) => candidate.name === `@askgina/${workspaceName}`, + ); + if (definition === undefined) { + return yield* fail(`unknown workspace dependency: @askgina/${workspaceName}`); + } + yield* fs + .symlink( + path.join(snapshot, definition.directory), + path.join(linkedScope, workspaceName), + ) + .pipe( + Effect.mapError((cause) => + fail(`cannot link workspace dependency ${definition.name}`, cause), + ), + ); + }), + ); + }), + ); + }); + yield* linkNodeModules("", true); + yield* Effect.forEach(PACKAGES, (definition) => linkNodeModules(definition.directory, false)); + const buildBin = path.join(snapshot, ".build-bin"); + yield* fs + .makeDirectory(buildBin) + .pipe(Effect.mapError((cause) => fail("cannot create snapshot build launcher", cause))); + yield* fs + .symlink(node, path.join(buildBin, "node")) + .pipe(Effect.mapError((cause) => fail("cannot link snapshot Node launcher", cause))); + yield* Effect.forEach(PACKAGES, (definition) => + runCommand(node, ["node_modules/.bin/vp", "pack", "--filter", definition.slug], snapshot, { + PATH: `${buildBin}:/usr/bin:/bin`, + }), + ); + }); + const assertLiveSourceBoundary = (root: string, snapshot: string) => Effect.gen(function* () { const fs = yield* FileSystem.FileSystem; @@ -397,7 +750,8 @@ const assertLiveSourceBoundary = (root: string, snapshot: string) => .readDirectory(live) .pipe(Effect.mapError((cause) => fail(`cannot list ${relative}`, cause))); yield* Effect.forEach(names.sort(), (name) => - name === "node_modules" + name === "node_modules" || + (name === "dist" && PACKAGES.some((definition) => definition.directory === relative)) ? Effect.void : visit(path.join(live, name), path.join(committed, name), `${relative}/${name}`), ); @@ -492,6 +846,7 @@ export const buildArtifacts = ({ root, dist }: BuildArtifactsOptions) => MAX_GIT_PORCELAIN_BYTES, ); if (sourceDirty) return yield* fail("artifact source tree must be clean"); + yield* buildSnapshotPackages(root, source); const contractSource = yield* readText(path.join(source, "packages/contracts/src/index.ts")); const catalogSha = contractSource.match(/export const catalogSha = "([a-f0-9]{64})"/u)?.[1]; const contractVersion = contractSource.match( @@ -517,7 +872,7 @@ export const buildArtifacts = ({ root, dist }: BuildArtifactsOptions) => const packageReceipts = yield* Effect.forEach(PACKAGES, (definition) => Effect.gen(function* () { const stage = path.join(temporary, `package-${definition.slug}`); - const files = yield* copyPackage(source, definition, stage, version); + const files = yield* stagePackage(root, source, definition.name, stage, version); const filename = `askgina-${definition.slug}-${version}.tgz`; const output = path.join(dist, "packages", filename); yield* archive(root, stage, output, ["package"]); @@ -530,6 +885,10 @@ export const buildArtifacts = ({ root, dist }: BuildArtifactsOptions) => }; }), ); + const contractFiles = yield* fileProofs( + path.join(source, "packages/contracts/dist"), + "packages/contracts/dist", + ); const sourceSkills = (yield* Effect.forEach(SKILLS, (skill) => fileProofs(path.join(plugin, "skills", skill), skill), @@ -640,9 +999,7 @@ export const buildArtifacts = ({ root, dist }: BuildArtifactsOptions) => sourceCommit, sourceDirty, catalogSha, - files: [ - { path: "packages/contracts/src/index.ts", sha256: yield* hash(contractSource) }, - ], + files: contractFiles, }, ], [ diff --git a/tools/verify-artifacts.ts b/tools/verify-artifacts.ts index 95b967d..32a39db 100755 --- a/tools/verify-artifacts.ts +++ b/tools/verify-artifacts.ts @@ -20,10 +20,10 @@ import { runHermeticEvalReplay, sanitizeEvalAggregate, sanitizeEvalReplay, -} from "../packages/evals/src/index.js"; -import { copyCheckedRegularFile, extractCheckedTarGz } from "./archive-security.js"; -import { checkGeneratedTargetConformance } from "./check-target-conformance.js"; -import { buildArtifacts } from "./pack-artifacts.js"; +} from "../packages/evals/src/index"; +import { copyCheckedRegularFile, extractCheckedTarGz } from "./archive-security"; +import { checkGeneratedTargetConformance } from "./check-target-conformance"; +import { buildArtifacts } from "./pack-artifacts"; const HOSTS = ["openai", "cursor", "claude", "copilot", "gemini"] as const; const SKILLS = [ @@ -63,11 +63,77 @@ const TARGET_MANIFESTS: Readonly> = { }; const SHA_256 = /^[a-f0-9]{64}$/u; const GIT_COMMIT = /^[a-f0-9]{40}$/u; +const NODE_24_VERSION = /^v24\.\d+\.\d+$/u; const SEMVER = /^(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)(?:-(?:(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?$/u; const MAX_GIT_PORCELAIN_BYTES = 64 * 1024; const RAW_EVAL_FIELDS = /"(?:prompts?|toolCalls?|tool_calls|payloads?|models?|accounts?|addresses?|final_answer|report)"\s*:/iu; +const NODE_24_SMOKE = String.raw` +import assert from "node:assert/strict"; +import { existsSync, readFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { Effect, Schema } from "effect"; +import { + GINA_READ_TOOL_CATALOG, + GinaReadToolCatalogSchema, + PRODUCTION_MCP_URL, +} from "@askgina/contracts"; +import { createClient, listCatalogToolNames } from "@askgina/sdk"; + +const packageRoot = (name) => dirname(dirname(fileURLToPath(import.meta.resolve(name)))); +for (const name of ["@askgina/contracts", "@askgina/sdk"]) { + const root = packageRoot(name); + assert.equal(existsSync(join(root, "src")), false, name + " installed raw source"); + const manifest = JSON.parse(readFileSync(join(root, "package.json"), "utf8")); + assert.deepEqual(Object.keys(manifest.exports["."]).sort(), ["import", "types"]); + assert.equal(manifest.type, "module"); + assert.equal(manifest.engines.node, ">=24"); +} + +const catalog = await Effect.runPromise( + Schema.decodeUnknownEffect(GinaReadToolCatalogSchema)(GINA_READ_TOOL_CATALOG), +); +assert.equal(catalog.length, 29); +const listed = GINA_READ_TOOL_CATALOG.map(({ name }) => ({ name })); +const client = createClient({ + accessToken: "offline-token", + transport: { + listTools: () => Effect.succeed(listed), + callTool: () => Effect.succeed({}), + }, +}); +assert.equal(client.url, PRODUCTION_MCP_URL); +assert.deepEqual( + (await Effect.runPromise(client.listTools())).map(({ name }) => name), + listCatalogToolNames(), +); + +let contacted = false; +const unauthenticated = createClient({ + accessToken: " ", + transport: { + listTools: () => Effect.sync(() => { contacted = true; return []; }), + callTool: () => Effect.sync(() => { contacted = true; return {}; }), + }, +}); +const authenticationError = await Effect.runPromise( + unauthenticated.listTools().pipe( + Effect.match({ onFailure: (error) => error, onSuccess: () => undefined }), + ), +); +assert.equal(authenticationError?._tag, "AskGinaAuthError"); +assert.equal(contacted, false); + +let stack = ""; +try { + createClient({ accessToken: "offline-token", url: "https://example.invalid" }); +} catch (error) { + stack = String(error?.stack ?? error); +} +assert.match(stack, /node_modules\/@askgina\/sdk\/src\/client\.ts:\d+:\d+/u); +`; type PackageDefinition = (typeof PACKAGES)[number]; type FileProof = Readonly<{ readonly path: string; readonly sha256: string }>; @@ -257,6 +323,57 @@ const runCommand = ( if (exitCode !== 0) return yield* fail(`${command} exited with ${exitCode}`); }), ); +export const runNodeEsmSmoke = ( + options: Readonly<{ + readonly node: string; + readonly cwd: string; + readonly source: string; + readonly env?: Record; + }>, +) => + runCommand( + options.node, + ["--enable-source-maps", "--input-type=module", "--eval", options.source], + options.cwd, + options.env, + ).pipe(Effect.mapError((cause) => fail("Node ESM smoke failed", cause))); + +export const verifyNoInstalledLibrarySources = (project: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + for (const name of ["contracts", "sdk"]) { + if (yield* fs.exists(path.join(project, "node_modules", "@askgina", name, "src"))) { + return yield* fail(`@askgina/${name} installed raw source`); + } + } + }); + +export const verifyNode24Consumer = ( + options: Readonly<{ + readonly node: string; + readonly project: string; + readonly env?: Record; + }>, +) => + Effect.gen(function* () { + const version = (yield* commandOutput( + options.node, + ["--version"], + options.project, + options.env, + )).trim(); + if (!NODE_24_VERSION.test(version)) { + return yield* fail(`Node 24 is required for package verification; found ${version}`); + } + yield* verifyNoInstalledLibrarySources(options.project); + yield* runNodeEsmSmoke({ + node: options.node, + cwd: options.project, + source: NODE_24_SMOKE, + env: options.env, + }); + }); const filesBelow = (directory: string) => Effect.gen(function* () { @@ -472,7 +589,7 @@ const verifyContract = ( const proofs = yield* parseProofs(receipt.files, "contract receipt files"); yield* verifyProofs( proofs, - [{ path: "packages/contracts/src/index.ts", sha256: yield* hash(source) }], + yield* fileProofs(path.join(root, "packages/contracts/dist"), "packages/contracts/dist"), "contract receipt", ); return receipt; @@ -650,14 +767,18 @@ const cleanInstall = ( env, ).pipe(Effect.mapError((cause) => fail("@askgina/evals omitted live adapters", cause))); const evalPackageRoot = path.join(project, "node_modules", "@askgina", "evals"); + if (yield* fs.exists(path.join(evalPackageRoot, "src"))) { + return yield* fail("@askgina/evals installed raw source"); + } + const evalSourceRoot = path.join(root, "packages/evals/src"); yield* runCommand( "bun", [ - path.join(evalPackageRoot, "src/bin/replay.ts"), + path.join(evalPackageRoot, "dist/bin/replay.js"), "--suite", - path.join(evalPackageRoot, "src/fixtures/model-smoke.yaml"), + path.join(evalSourceRoot, "fixtures/model-smoke.yaml"), "--observations", - path.join(evalPackageRoot, "src/fixtures/synthetic-observations.yaml"), + path.join(evalSourceRoot, "fixtures/synthetic-observations.yaml"), ], project, env, @@ -773,6 +894,17 @@ const verifyPackages = ( .pipe(Effect.mapError((cause) => fail(`cannot access ${archive}`, cause))); }), ); + const nodeProject = path.join(temporary, "install-sdk"); + yield* verifyNode24Consumer({ + node: "node", + project: nodeProject, + env: { + PATH: yield* Config.string("PATH").pipe(Config.withDefault("/usr/bin:/bin")), + HOME: path.join(nodeProject, ".home"), + LC_ALL: "C", + TZ: "UTC", + }, + }); }); const verifyTargets = ( diff --git a/tsconfig.json b/tsconfig.json index 8d285e8..286f1cf 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -1,10 +1,10 @@ { "compilerOptions": { "strict": true, - "target": "ES2022", - "lib": ["ES2022"], - "module": "NodeNext", - "moduleResolution": "NodeNext", + "target": "ESNext", + "lib": ["ESNext"], + "module": "Preserve", + "moduleResolution": "Bundler", "verbatimModuleSyntax": true, "isolatedModules": true, "noEmit": true, diff --git a/vite.config.ts b/vite.config.ts index a179724..fdfdead 100644 --- a/vite.config.ts +++ b/vite.config.ts @@ -1,7 +1,68 @@ import { recommended as effectTsgoRecommended } from "@effect/tsgo/oxlint-presets"; +import { fileURLToPath } from "node:url"; import { defineConfig } from "vite-plus"; +import type { PackUserConfig } from "vite-plus/pack"; + +const packDefaults = { + deps: { neverBundle: true }, + dts: true, + fixedExtension: false, + format: "esm", + plugins: [ + { + name: "strip-unemitted-declaration-map-references", + generateBundle(_options, bundle) { + for (const output of Object.values(bundle)) { + if (output.type === "chunk" && output.fileName.endsWith(".d.ts")) { + output.code = output.code.replace( + /\n?\/\/# sourceMappingURL=[^\r\n]+\.d\.ts\.map\s*$/u, + "\n", + ); + } + } + }, + }, + ], + sourcemap: true, +} satisfies PackUserConfig; export default defineConfig({ + pack: [ + { + ...packDefaults, + cwd: fileURLToPath(new URL("packages/contracts/", import.meta.url)), + entry: ["src/index.ts"], + name: "contracts", + }, + { + ...packDefaults, + cwd: fileURLToPath(new URL("packages/sdk/", import.meta.url)), + entry: ["src/index.ts"], + name: "sdk", + }, + { + ...packDefaults, + cwd: fileURLToPath(new URL("packages/cli/", import.meta.url)), + entry: { bin: "bin.ts", index: "src/index.ts" }, + name: "cli", + }, + { + ...packDefaults, + cwd: fileURLToPath(new URL("packages/evals/", import.meta.url)), + entry: { + "bin/live": "src/bin/live.ts", + "bin/replay": "src/bin/replay.ts", + index: "src/index.ts", + }, + name: "evals", + }, + { + ...packDefaults, + cwd: fileURLToPath(new URL("plugins/ask-gina/", import.meta.url)), + entry: ["src/index.ts"], + name: "plugin-core", + }, + ], fmt: { ignorePatterns: ["dist/**"], }, @@ -30,9 +91,22 @@ export default defineConfig({ cache: { scripts: false, tasks: true }, tasks: { quality: { - command: ["vp check .", "bun run typecheck"], + command: [ + "vp check .", + "node_modules/.bin/tsc --noEmit -p tsconfig.json", + "bun run check:typescript-imports", + ], + dependsOn: ["build-packages"], output: [], }, + "build-packages": { + command: "vp run --filter '@askgina/*' --fail-if-no-match build", + input: [{ auto: true }, "!packages/*/dist/**", "!plugins/*/dist/**"], + output: [ + { pattern: "packages/*/dist/**", base: "workspace" }, + { pattern: "plugins/*/dist/**", base: "workspace" }, + ], + }, tests: { command: "bun --bun node_modules/.bin/vp test --run", dependsOn: ["quality"],