Skip to content

SPEC.md

Latest commit

 

History

History
217 lines (169 loc) · 5.08 KB

SPEC.md

File metadata and controls

217 lines (169 loc) · 5.08 KB
Raw

perf full specification

Objective

Build a local Node.js v24+ ESM CLI named perf that captures Chrome runtime performance traces for deterministic app flows, analyzes those traces into actionable findings, and emits compact files that Codex can use to fix performance problems.

perf should make this workflow repeatable:

run app
open Chrome
execute scenario
record trace
analyze slow work
write findings
write Codex fix packet
external manager applies fix
external manager reruns perf

Product shape

perf is a measurement and reporting tool. It should be deterministic, scriptable, and friendly to automation.

It should not become a general-purpose test runner, browser automation framework, agent framework, or patch applier.

Required capabilities

  • Load perf.config.mjs.
  • Launch or attach to a local app server.
  • Launch Chrome through Playwright.
  • Use Chrome DevTools Protocol through a Playwright CDP session.
  • Capture raw trace JSON with streamed output.
  • Execute deterministic browser scenarios.
  • Mark each action in the trace with performance.mark() and performance.measure().
  • Inject observers for long tasks, event timings, layout shifts, and user timing.
  • Write stable artifacts under .perf/runs/<run-id>/<scenario>/.
  • Normalize noisy trace data into perf-findings.json.
  • Write perf-summary.json for external automation.
  • Gate configurable budgets.
  • Generate perf-report.md for humans.
  • Generate codex-fix-packet.md for coding agents.
  • Optionally compare two runs.

Non-goals

  • Do not own the agent repair loop.
  • Do not call Codex directly in the MVP.
  • Do not paste raw traces into Codex prompts.
  • Do not require Lighthouse.
  • Do not depend on Express, Axios, Python, Java, CommonJS, or dotenv.
  • Do not hide app build/test failures behind perf reports.

Runtime requirements

  • Node.js v24+.
  • ESM only.
  • async/await only for asynchronous control flow.
  • Node core imports must use node: prefixes.
  • Environment loading should be done by the caller with node --env-file=.env.
  • Chrome automation should use Playwright.
  • CDP trace capture should use Playwright CDPSession.

Package name

Use:

packages/perf

Recommended published name:

@async/perf

CLI binary:

perf

Commands

perf init
perf run
perf analyze
perf gate
perf packet
perf compare
perf report
perf list-scenarios

The command set intentionally excludes perf loop. A higher-level manager owns the repeated repair loop.

File namespace policy

.perf/                         measured records
.async/perf/                   ignored perf tool state
.async/locks/perf/             @async/lock runtime leases, if locking is used

Do not create random dot folders like .perfloop, .chrome-perf, .cache/perf, .tmp/perf, or .codex-perf.

Artifact layout

.perf/runs/<run-id>/
  perf-summary.json
  run-meta.json
  <scenario>/
    trace.json
    trace-meta.json
    browser-console.json
    event-timing.json
    long-tasks.json
    layout-shifts.json
    user-timing.json
    performance-metrics-before.json
    performance-metrics-after.json
    perf-findings.json
    perf-report.md
    codex-fix-packet.md

Use .perf/runs/latest as a symlink when the platform supports it. If symlinks are unavailable, write .perf/runs/latest.json with the latest run path.

Primary performance targets

perf should diagnose runtime interaction problems:

  • Long tasks.
  • Slow clicks and inputs.
  • Expensive event handlers.
  • Expensive route transitions.
  • Forced layout and style recalculation.
  • Heavy paint and compositing.
  • Hydration/resume/render cost.
  • Main-thread blocking network-adjacent work.
  • Repeated regressions between runs.

Default budgets

export const defaultBudgets = {
  maxLongTaskMs: 100,
  maxBlockingMsPerAction: 100,
  maxTotalBlockingMsPerScenario: 300,
  maxInteractionMs: 200,
  maxLayoutMsPerAction: 50,
  maxScriptMsPerAction: 150,
  maxRegressionPercent: 10
};

Finding model

Each finding must be ranked, scoped to a scenario step where possible, and linked to evidence.

Required fields:

id
kind
severity
scenario
step
selector or url
metric values
repeatability
trace window
top events
top source locations when available
likely cause
suggested fix direction
evidence paths

Codex packet rule

The packet should include enough evidence for a coding agent to patch the app, but it must not include the full trace.

Include:

  • Objective.
  • Repro command.
  • Failing budgets.
  • Top findings.
  • Top source files/functions.
  • Likely cause.
  • Fix direction.
  • Constraints.
  • Verification commands.

Reference raw trace paths instead of inlining trace JSON.

Async pipeline rule

When building automation around perf, use async/pipeline runtime code. Do not add a bespoke internal loop to perf.

Example high-level pipeline:

checkout/build app
run perf
read .perf/runs/latest/perf-summary.json
if failed, publish codex-fix-packet.md
external Codex step patches code
rerun tests/build/perf
compare baseline vs latest

See docs/11-async-pipeline-runtime.md and examples/pipelines/.