Merge remote-tracking branch 'origin/main' into fix/package-repo-tiebreak

Signed-off-by: anilb <epipav@gmail.com>

# Conflicts:
#	services/libs/data-access-layer/src/osspckgs/api.ts

Author: epipav

.claude/rules/skill-guidance.md

@@ -18,6 +18,8 @@ This project has guided skills for common workflows. **Proactively suggest the r
 | `/scaffold-snowflake-connector` | Add a new Snowflake-connector data source or integration                          |
 | `/packages-worker-setup` | First-time setup of packages-db and github-repos-enricher for a new engineer    |
 | `/packages-worker-add-entrypoint` | Scaffold a new sibling worker inside packages_worker (npm, OSV, scorecard, etc.) |
+| `/write-unit-tests` | Add or improve Vitest unit tests for business logic, DAL, or server modules |
+| `/write-api-e2e-tests` | Add or change Public API e2e / smoke / contract tests |
 
 ## Trigger Phrases
 
@@ -57,3 +59,14 @@ This project has guided skills for common workflows. **Proactively suggest the r
 - "Add a new packages worker", "scaffold a sibling worker", "new entry point in packages_worker"
 - "Add npm ingestion", "add OSV worker", "add scorecard runner"
 - Any request to create a new `src/bin/*.ts` worker inside `packages_worker`
+
+**`/write-unit-tests`** — match any of these intents:
+- "Write unit tests", "add a unit test", "cover this with Vitest"
+- "Test this function", "add DAL tests", "unit test for affiliations/merges/inference"
+- Any request for focused server/unit test coverage (not HTTP/API contract tests)
+- Not for `packages_worker` (excluded from `pnpm test:server` for now)
+
+**`/write-api-e2e-tests`** — match any of these intents:
+- "Write API e2e tests", "add smoke tests", "Public API contract tests"
+- "Cover this endpoint end-to-end", "API regression test"
+- Any request for HTTP/API e2e coverage of Public API behaviour

.claude/skills/write-api-e2e-tests/SKILL.md

@@ -0,0 +1,66 @@
+---
+name: write-api-e2e-tests
+description: >
+  Write API end-to-end tests. Use when adding or changing API endpoints, or when
+  the user asks for API e2e, smoke, or contract tests.
+allowed-tools: Bash, Read, Glob, Grep, Edit, Write, AskUserQuestion
+---
+
+# Write API end-to-end tests
+
+Write or extend API end-to-end tests.
+
+## When to use
+
+- New or changed API endpoints.
+- User asks for API e2e, smoke, or contract tests.
+- Critical API behaviour needs regression coverage.
+
+## When not to use
+
+- Domain or SQL correctness → use `write-unit-tests`.
+- Temporal, OpenSearch, or other eventual side effects outside the documented API e2e scope.
+- Creating a new testing framework or suite style.
+
+## Source of truth
+
+Read before writing. Follow these ADRs and existing suite structure; do not
+invent a parallel testing style.
+
+- [ADR-0012](../../../docs/adr/0012-api-e2e-test-architecture.md) — runtime, isolation, supported surfaces, scope, and assertions.
+- [ADR-0013](../../../docs/adr/0013-api-e2e-test-suite-design.md) — suite organisation, helpers, and conventions.
+
+Current default entrypoint:
+
+- `.github/scripts/public-api-e2e-tests.sh`
+
+## Workflow
+
+1. Identify the API surface. Default to Public API unless the user specifies otherwise.
+2. Read ADR-0012 and ADR-0013.
+3. Add or extend the appropriate suite and register it if required.
+4. Run the affected suite locally and fix failures until green.
+5. If required fixtures cannot be created through the API, prefer testing supported scenarios and explain any coverage gaps instead of seeding the database directly.
+6. Suggest production testability improvements only when they make the API easier to test, and ask before changing production code.
+
+## Run
+
+Export the environment variables required by the suite entrypoint.
+
+```bash
+bash .github/scripts/public-api-e2e-tests.sh
+```
+
+Refer to ADR-0012 and the suite entrypoint for environment setup, reset behaviour,
+and local development workflows.
+
+## Guardrails
+
+- Keep tests focused on observable API behaviour.
+
+## Output
+
+- Suites and cases added
+- How to re-run
+- Coverage gaps, if any
+- Optional testability suggestions

.claude/skills/write-unit-tests/SKILL.md

@@ -0,0 +1,72 @@
+---
+name: write-unit-tests
+description: >
+  Write focused Vitest unit tests. Use when adding or improving unit tests for
+  business logic, data access, common services, or other server modules.
+allowed-tools: Bash, Read, Glob, Grep, Edit, Write, AskUserQuestion
+---
+
+# Write unit tests
+
+Write focused unit tests that follow the project's testing conventions.
+
+## When to use
+
+- User asks to add or improve unit tests.
+- A change touches high-blast-radius logic (affiliations, merges, identity resolution, timelines, inference).
+- A PR needs confidence in a pure or Postgres-backed function.
+
+## When not to use
+
+- Public or HTTP contract coverage → use `write-api-e2e-tests`.
+- `packages_worker` (excluded from `pnpm test:server` for now; own vitest/packages-db).
+- Temporal, Redis, or OpenSearch fixtures (not available yet).
+- Broad "increase coverage %" requests without a clear unit under test.
+
+## Source of truth
+
+Read before writing. Follow these ADRs; do not invent a parallel testing style.
+
+- [ADR-0008](../../../docs/adr/0008-how-we-write-unit-tests.md) — scenarios, `describe` grouping, assertions, mocking, and shared setup.
+- [ADR-0007](../../../docs/adr/0007-test-factory-primitives-and-defaults.md) — factories and defaults.
+
+## Workflow
+
+1. Identify the unit under test (one function or decision path). Colocate tests as `<file>.test.ts`.
+2. Read ADR-0007 and ADR-0008. Skim the nearest existing test in the same area if one exists.
+3. Compose fixtures using `@crowd/test-kit` (`withQx` for Postgres-backed tests; factories and opt-in defaults per ADR-0007).
+4. Write focused scenarios following ADR-0008 (grouping, naming, assertions, and mocking).
+5. Run the affected tests and fix failures until green.
+6. If production code is difficult to test, suggest a small testability seam and ask before changing production code.
+
+## Run
+
+Start the test database when needed:
+
+```bash
+./scripts/cli scaffold up-test
+```
+
+Run a focused test file:
+
+```bash
+pnpm test:server -- path/to/file.test.ts
+```
+
+Optional:
+
+```bash
+pnpm test:changed
+pnpm test:watch -- path/to/file.test.ts
+```
+
+## Guardrails
+
+- Prefer critical behaviours over trivial getters, setters, and thin wrappers.
+- Keep production behaviour unchanged unless the user explicitly asks for a testability improvement.
+
+## Output
+
+- Scenarios covered
+- How to re-run
+- Optional testability suggestions

.env.test

@@ -0,0 +1,5 @@
+NODE_ENV=test
+DB_HOST=localhost
+DB_PORT=5434
+DB_USER=postgres
+DB_PASSWORD=example

.github/scripts/notify-api-e2e-failure.sh

@@ -0,0 +1,68 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+: "${SLACK_WEBHOOK_URL:?}"
+: "${RUN_URL:?}"
+: "${EVENT_NAME:?}"
+: "${ALERT_TITLE:?}"
+
+failed_step() {
+  local pairs=(
+    "${OUTCOME_RESOLVE_TAG}:Resolve deploy image tag"
+    "${OUTCOME_DEPLOY}:Deploy api-e2e"
+    "${OUTCOME_HEALTH}:Wait for api-e2e service to be ready"
+    "${OUTCOME_E2E}:Run e2e tests"
+  )
+  local pair outcome label
+  for pair in "${pairs[@]}"; do
+    outcome="${pair%%:*}"
+    label="${pair#*:}"
+    if [[ "$outcome" == "failure" ]]; then
+      printf '%s\n' "$label"
+      return
+    fi
+  done
+  printf 'unknown\n'
+}
+
+read_summary() {
+  local path="${E2E_SUMMARY_PATH:-e2e.summary}"
+  results=""
+  failures=""
+
+  [[ -f "$path" ]] || return 0
+
+  local passed failed
+  passed="$(awk -F= '/^passed=/{print $2; exit}' "$path")"
+  failed="$(awk -F= '/^failed=/{print $2; exit}' "$path")"
+  if [[ -n "${passed}" || -n "${failed}" ]]; then
+    results="Passed: ${passed:-?} / Failed: ${failed:-?}"
+  fi
+  failures="$(awk '/^FAIL /{print; if (++n == 10) exit}' "$path")"
+}
+
+sha_short="${DEPLOY_TAG:-unknown}"
+sha_short="${sha_short:0:12}"
+
+read_summary
+
+lines=(
+  ":rotating_light: *${ALERT_TITLE}*"
+  "*Event:* \`${EVENT_NAME}\`"
+  "*Failed step:* \`$(failed_step)\`"
+  "*Deploy SHA:* \`${sha_short}\`"
+  "*Run:* <${RUN_URL}|View run>"
+)
+
+[[ -n "${results}" ]] && lines+=("*Results:* ${results}")
+
+if [[ -n "${failures}" ]]; then
+  lines+=("*Failed tests:*" $'```\n'"${failures}"$'\n```')
+fi
+
+text="$(printf '%s\n' "${lines[@]}")"
+payload="$(jq -n --arg text "$text" '{text: $text}')"
+
+curl -fsS -X POST "$SLACK_WEBHOOK_URL" \
+  -H 'Content-Type: application/json' \
+  -d "$payload"

.github/scripts/prepare-test-template-db.sh

@@ -0,0 +1,45 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+# shellcheck source=scripts/utils
+source "${REPO_ROOT}/scripts/utils"
+
+: "${DB_HOST:?DB_HOST is required}"
+: "${DB_PORT:?DB_PORT is required}"
+: "${DB_USER:?DB_USER is required}"
+: "${DB_PASSWORD:?DB_PASSWORD is required}"
+
+say "Applying Sequin bootstrap SQL..."
+docker run --rm --network host \
+  -v "${REPO_ROOT}/scripts/scaffold/sequin/postgres-docker-entrypoint-initdb.d/create-sequin-database.sql:/bootstrap.sql:ro" \
+  -e "PGPASSWORD=${DB_PASSWORD}" \
+  postgres:14-alpine \
+  psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -v ON_ERROR_STOP=1 -f /bootstrap.sql
+
+say "Recreating test_template..."
+docker run --rm --network host \
+  -e "PGPASSWORD=${DB_PASSWORD}" \
+  postgres:14-alpine \
+  psql -h "$DB_HOST" -p "$DB_PORT" -U "$DB_USER" -d postgres -v ON_ERROR_STOP=1 \
+  -c "DROP DATABASE IF EXISTS test_template;" \
+  -c "CREATE DATABASE test_template;"
+
+if [[ "${SKIP_FLYWAY_BUILD:-0}" == "1" ]]; then
+  say "Using pre-built crowd_flyway image."
+else
+  say "Building flyway image..."
+  docker build -t crowd_flyway -f "${REPO_ROOT}/backend/src/database/Dockerfile.flyway" "${REPO_ROOT}/backend/src/database"
+fi
+
+say "Migrating test_template..."
+docker run --rm --network host \
+  -e "PGHOST=${DB_HOST}" \
+  -e "PGPORT=${DB_PORT}" \
+  -e "PGUSER=${DB_USER}" \
+  -e "PGPASSWORD=${DB_PASSWORD}" \
+  -e PGDATABASE=test_template \
+  crowd_flyway
+
+say "Test template database ready."