Tests

fix feedback tune scorer judge prompt #116

Workflow file for this run

	name: Tests
	# The engine and judge-core, the Go CLI, and the engine again against Postgres - the backend where
	# concurrent writes actually interleave (engine/src/test/dialect.integration.test.ts).
	on:
	push:
	branches: [main]
	pull_request:
	workflow_dispatch: {}

	# Read-only: nothing here publishes. Same explicit block release.yml and publish-judge-core.yml
	# already set, and what CodeQL's workflow-permissions check asks for.
	permissions:
	contents: read

	jobs:
	typescript:
	name: engine + judge-core
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1

	- name: Set up Node.js
	uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020
	with:
	node-version: 24.x
	cache: yarn
	cache-dependency-path: yarn.lock

	- name: Install workspace dependencies
	run: yarn install --frozen-lockfile

	# The engine imports judge-core through its "exports" map, so it has to be built first.
	- name: Build judge-core
	run: yarn workspace @agentx/judge-core build

	- name: Typecheck
	run: \|
	yarn workspace @agentx/judge-core typecheck
	yarn workspace @agentx/engine typecheck
	yarn workspace @agentx/eval typecheck

	# Async discipline the compiler can't enforce: no-floating-promises et al. - see
	# engine/eslint.config.mjs for why the rule set is deliberately narrow.
	- name: Lint engine
	run: yarn workspace @agentx/engine lint

	- name: Test judge-core
	run: yarn workspace @agentx/judge-core test

	# The TypeScript CI SDK pins the /custom-agent-evaluations wire contract from the
	# consumer side (stub-server tests, no engine needed).
	- name: Test @agentx/eval
	run: yarn workspace @agentx/eval build && yarn workspace @agentx/eval test

	# The integration suites boot the real engine per file, so this outruns the job default.
	# Coverage is a RATCHET, not a target: c8 merges the spawned engine processes' V8 coverage
	# (they exit gracefully on SIGTERM, so it flushes) and the floors sit under the measured
	# baseline (45% lines / 30% functions at introduction). Raise the floors when coverage
	# rises; never lower them to make a PR pass.
	- name: Test engine (with coverage ratchet)
	run: yarn workspace @agentx/engine test:coverage
	timeout-minutes: 20

	postgres:
	name: engine on Postgres
	runs-on: ubuntu-latest
	services:
	postgres:
	image: postgres:16-alpine
	env:
	POSTGRES_PASSWORD: agentx
	POSTGRES_DB: agentx
	ports:
	- 5432:5432
	options: >-
	--health-cmd pg_isready
	--health-interval 10s
	--health-timeout 5s
	--health-retries 5
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1

	- name: Set up Node.js
	uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020
	with:
	node-version: 24.x
	cache: yarn
	cache-dependency-path: yarn.lock

	- name: Install workspace dependencies
	run: yarn install --frozen-lockfile

	- name: Build judge-core
	run: yarn workspace @agentx/judge-core build

	# AGENTX_TEST_DB_URL un-skips the Postgres suites; they create and drop their own databases.
	- name: Test engine against Postgres
	run: yarn workspace @agentx/engine test
	timeout-minutes: 20
	env:
	AGENTX_TEST_DB_URL: postgres://postgres:agentx@localhost:5432/agentx

	cli:
	name: cli
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1

	# Tag, not a pinned SHA: the SHAs above are carried over from this repo's other workflows,
	# and this action is new here. cli/ has no dependencies, so there is no go.sum to cache on.
	- name: Set up Go
	uses: actions/setup-go@v7
	with:
	go-version-file: cli/go.mod
	cache: false

	# gofmt is the whole formatting standard for Go, so "we all run gofmt" costs nothing to
	# enforce and stops the first unformatted file from turning every later diff into noise.
	# `gofmt -l` lists offenders and exits 0 either way, hence the explicit emptiness check.
	- name: Format
	working-directory: cli
	run: \|
	unformatted=$(gofmt -l .)
	if [ -n "$unformatted" ]; then
	echo "These files are not gofmt-clean. Run: gofmt -w ." >&2
	echo "$unformatted" >&2
	gofmt -d .
	exit 1
	fi

	- name: Vet
	working-directory: cli
	run: go vet ./...

	# -race, because the CLI's whole job is process supervision: cmd/server.go spawns the engine
	# and juggles its stdio, signals, and shutdown across goroutines, which is exactly the shape
	# of code where a data race hides behind a passing test. The suite runs in seconds, so the
	# detector's overhead buys a real class of bug for no meaningful time.
	- name: Test
	working-directory: cli
	run: go test -race ./...

	shell:
	name: shell scripts
	# install.sh is piped straight into bash from a curl one-liner in the README, and build.sh and
	# scripts/ are what a contributor runs to produce a real distribution. A quoting bug in any of
	# them fails on someone else's machine, after the merge. shellcheck is preinstalled on the
	# GitHub runner image, so this is a check with no install step and no new action to pin.
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1

	- name: Shellcheck
	run: shellcheck install.sh build.sh scripts/*.sh

	binary:
	name: compiled binary
	# The suites above run under tsx, which storage/db.ts serves with better-sqlite3. The artifact
	# release.yml ships is `bun build --compile`, which takes the bun:sqlite branch instead - a
	# driver nothing else here covers. Smoke-only: boot, write, read, shut down.
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1

	- name: Set up Node.js
	uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020
	with:
	node-version: 24.x
	cache: yarn
	cache-dependency-path: yarn.lock

	- name: Set up Bun
	uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6
	with:
	bun-version: "1.3.14"

	- name: Install workspace dependencies
	run: yarn install --frozen-lockfile

	# `bun build` resolves @agentx/judge-core through its dist/, which a fresh checkout lacks.
	- name: Build judge-core
	run: yarn workspace @agentx/judge-core build

	- name: Compile the engine
	run: bun build engine/src/index.ts --compile --outfile /tmp/agentx-engine

	- name: Smoke-test the binary
	run: ./scripts/smoke-binary.sh /tmp/agentx-engine

Navigation Menu

Uh oh!

fix feedback tune scorer judge prompt #116

Workflow file

fix feedback tune scorer judge prompt #116

Uh oh!

Workflow file for this run