Skip to content

Commit 00d0b50

Browse files
kriszypkriszyp
authored
Merge branch 'main' into kris/dbis-records-mode-downgrade
2 parents 7d13b27 + 96162c7 commit 00d0b50

120 files changed

Lines changed: 16700 additions & 19781 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/claude-review.yml‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,14 @@ concurrency:
2626

2727
jobs:
2828
review:
29-
uses: HarperFast/ai-review-prompts/.github/workflows/_claude-review.yml@2be0f702b114e39b90eed90962922ef64df58482 # main 2026-06-03 (post #52 — week-of-05-25 calibration: Harper-aware best-practices wiring, skills-ref v1.7.0, surface-findings rule, access-control blocker guidance, public-skill dedup)
29+
# Always-on toggle — see ai-review-prompts USAGE.md "Reviewers & the
30+
# always-on toggle". CLAUDE_ALWAYS_ON=true (repo/org variable) → auto-
31+
# review trusted-author PRs; unset → opt-in via the claude-review
32+
# label. The reusable's authorize job still owns WHO is admitted.
33+
# Note: the `claude-review` label name is matched there too —
34+
# `_claude-review.yml`'s authorize `if:`, not in this caller.
35+
if: ${{ vars.CLAUDE_ALWAYS_ON == 'true' || github.event.action == 'labeled' }}
36+
uses: HarperFast/ai-review-prompts/.github/workflows/_claude-review.yml@3f2300a39910a0983eb74967dedeebd8c3c98adc # main 2026-06-08 (post #57 — week-of-06-01 calibration: severity-discipline non-blocker bullets, async-commit-ordering check, non-critical-work-on-critical-path guard)
3037
with:
3138
# Same SHA as the `uses:` ref above. The reusable uses this to
3239
# check out HarperFast/ai-review-prompts (layer files + bash
@@ -37,7 +44,7 @@ jobs:
3744
# introspect their own ref (`github.workflow_ref` resolves to the
3845
# CALLER's ref in `workflow_call` context), and `uses: …@<ref>`
3946
# is parsed literally so we can't interpolate a variable.
40-
ai-review-prompts-ref: 2be0f702b114e39b90eed90962922ef64df58482
47+
ai-review-prompts-ref: 3f2300a39910a0983eb74967dedeebd8c3c98adc
4148
review-layers: |
4249
universal
4350
harper/common

‎.github/workflows/gemini-review.yml‎

Lines changed: 108 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,108 @@
1+
name: Gemini PR Review
2+
3+
# Thin caller of the Gemini reusable in HarperFast/ai-review-prompts.
4+
# Runs in parallel with claude-review.yml so the two reviewers can be
5+
# compared on the same PRs.
6+
#
7+
# Layer inputs and `repo-specific-checks:` MIRROR claude-review.yml in
8+
# this repo. Output comparability between the two providers depends on
9+
# them seeing the same review scope — keep them in sync when bumping
10+
# the pin or editing the checks block.
11+
#
12+
# Opt-in by default: Gemini runs only when a HarperFast org member
13+
# applies the `gemini-review` label, UNLESS the GEMINI_ALWAYS_ON repo
14+
# variable is set to 'true' (then it auto-reviews trusted-author PRs,
15+
# like claude-review). See ai-review-prompts USAGE.md "Reviewers & the
16+
# always-on toggle".
17+
#
18+
# Pre-requisites:
19+
# - HARPERFAST_AI_CLIENT_ID (org-level App Client ID)
20+
# - HARPERFAST_AI_APP_PRIVATE_KEY (org-level App private key)
21+
# - GEMINI_API_KEY (per-repo; optional — a missing key
22+
# cleanly skips the review with a
23+
# workflow notice, so this is safe to
24+
# merge before the key is set)
25+
# - AI_REVIEW_LOG_TOKEN (optional — threads each run into a
26+
# per-(PR, provider) issue in
27+
# HarperFast/ai-review-log with the
28+
# `provider:gemini` label)
29+
30+
on:
31+
pull_request:
32+
# `labeled` admits the `gemini-review` opt-in gesture. `vars.*`
33+
# can't be read in `on:` (only in a job `if:`), so the trigger
34+
# lists the union and the `review` job gates on GEMINI_ALWAYS_ON.
35+
types: [opened, synchronize, reopened, labeled]
36+
37+
concurrency:
38+
# Different group key from claude-review so the two providers can run
39+
# in parallel on the same PR. cancel-in-progress is per-group, so a
40+
# synchronize push cancels the in-flight Gemini run without touching
41+
# the Claude run (and vice versa).
42+
group: gemini-review-${{ github.event.pull_request.number }}
43+
cancel-in-progress: true
44+
45+
jobs:
46+
review:
47+
# Always-on toggle — see ai-review-prompts USAGE.md "Reviewers & the
48+
# always-on toggle". GEMINI_ALWAYS_ON=true (repo/org variable) → auto-
49+
# review trusted-author PRs; unset → opt-in via the gemini-review
50+
# label. The reusable's authorize job still owns WHO is admitted
51+
# (CODEOWNERS trust set; the labeler, not the author, on `labeled`).
52+
# Note: the `gemini-review` label name is matched there too —
53+
# `_gemini-review.yml`'s authorize `if:`, not in this caller.
54+
if: ${{ vars.GEMINI_ALWAYS_ON == 'true' || github.event.action == 'labeled' }}
55+
uses: HarperFast/ai-review-prompts/.github/workflows/_gemini-review.yml@3f2300a39910a0983eb74967dedeebd8c3c98adc # main 2026-06-08 (post #57 — week-of-06-01 calibration: severity-discipline non-blocker bullets, async-commit-ordering check, non-critical-work-on-critical-path guard)
56+
# Caller-side permissions at the calling-job level (NOT workflow-
57+
# level — that placement caps the reusable's per-job grants below
58+
# what they need and breaks the workflow at startup; see
59+
# ai-review-prompts#39/#40). Union of what the reusable's authorize
60+
# (`contents: read`) and review (`contents: read` + `pull-requests:
61+
# write` + `id-token: write`) jobs declare.
62+
permissions:
63+
contents: read
64+
pull-requests: write
65+
id-token: write
66+
with:
67+
# Same SHA as the `uses:` ref above. The reusable uses this to
68+
# check out HarperFast/ai-review-prompts (layer files + bash
69+
# scripts) at the same ref as the workflow logic itself. The
70+
# duplication is unavoidable: reusable workflows can't introspect
71+
# their own ref (`github.workflow_ref` resolves to the CALLER's
72+
# ref in workflow_call context), and `uses: …@<ref>` is literal.
73+
ai-review-prompts-ref: 3f2300a39910a0983eb74967dedeebd8c3c98adc
74+
review-layers: |
75+
universal
76+
harper/common
77+
harper/v5
78+
repo-specific-checks: |
79+
## Repo-specific checks (Harper core)
80+
81+
This repo IS Harper core itself — "defer to Harper docs"
82+
guidance from the layers applies to PLUGIN / APP docs,
83+
not to docs within this repo (this is where the Harper docs'
84+
behavior is defined).
85+
86+
On top of the layered scope, these are things specific to this
87+
repo that the shared layers don't cover:
88+
89+
- **Linter is oxlint, not eslint.** `npm run lint` runs oxlint.
90+
Advice in layers that references ESLint doesn't apply here.
91+
- **Build tolerance (`tsc || true`)** is NOT used here —
92+
Harper core's build should pass cleanly. Flag type errors
93+
as real findings.
94+
- **`dependencies.md`** documents all npm packages. New
95+
runtime dependencies require an entry there; flag PRs that
96+
add a dep without updating the file.
97+
- **TypeStrip compatibility** — Harper core uses
98+
`erasableSyntaxOnly`. Flag TypeScript constructs that would
99+
break typestrip (non-type-only imports of types, parameter
100+
property initialization, etc.).
101+
- **RocksDB is primary storage** (LMDB still supported via
102+
`HARPER_STORAGE_ENGINE=lmdb`). Tests should exercise the
103+
primary path; flag PRs that test only the fallback.
104+
secrets:
105+
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
106+
AI_REVIEW_LOG_TOKEN: ${{ secrets.AI_REVIEW_LOG_TOKEN }}
107+
HARPERFAST_AI_CLIENT_ID: ${{ secrets.HARPERFAST_AI_CLIENT_ID }}
108+
HARPERFAST_AI_APP_PRIVATE_KEY: ${{ secrets.HARPERFAST_AI_APP_PRIVATE_KEY }}

‎.github/workflows/integration-tests.yml‎

Lines changed: 0 additions & 186 deletions
Original file line numberDiff line numberDiff line change
@@ -74,192 +74,6 @@ jobs:
7474
package.json
7575
retention-days: 1
7676

77-
run-integration-apiTests:
78-
name: Integration API Tests (Node.js v${{ matrix.node-version }})
79-
needs: [generate-node-version-matrix, build]
80-
runs-on: ubuntu-latest
81-
timeout-minutes: 15
82-
strategy:
83-
fail-fast: false
84-
matrix:
85-
node-version: ${{ fromJSON(needs.generate-node-version-matrix.outputs.node-versions) }}
86-
87-
steps:
88-
- name: Checkout code
89-
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
90-
91-
- name: Setup Node.js ${{ matrix.node-version }}
92-
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
93-
with:
94-
node-version: ${{ matrix.node-version }}
95-
package-manager-cache: false
96-
97-
- name: Download build artifacts
98-
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
99-
with:
100-
name: harper-build-artifacts-node-${{ matrix.node-version }}
101-
102-
- name: Setup Harper
103-
env:
104-
DEFAULTS_MODE: 'dev'
105-
HDB_ADMIN_USERNAME: 'admin'
106-
HDB_ADMIN_PASSWORD: 'password'
107-
ROOTPATH: '/tmp/hdb'
108-
OPERATIONSAPI_NETWORK_PORT: 9925
109-
LOGGING_LEVEL: 'debug'
110-
LOGGING_STDSTREAMS: true
111-
THREADS_COUNT: 1
112-
THREADS_DEBUG: false
113-
NODE_HOSTNAME: 'localhost'
114-
run: |
115-
mkdir -p /tmp/hdb/log
116-
node ./dist/bin/harper.js install > /tmp/hdb/log/install-stdout.log 2> /tmp/hdb/log/install-stderr.log
117-
node ./dist/bin/harper.js start > /tmp/hdb/log/start-stdout.log 2> /tmp/hdb/log/start-stderr.log &
118-
echo "Waiting for Harper to be ready on port 9925..."
119-
timeout 120 bash -c 'until nc -z localhost 9925 2>/dev/null; do sleep 0.5; done'
120-
echo "Harper is ready."
121-
122-
- name: Run API Tests
123-
id: run-api-tests
124-
env:
125-
HDB_ADMIN_USERNAME: 'admin'
126-
HDB_ADMIN_PASSWORD: 'password'
127-
run: npm run test:integration:api-tests
128-
129-
- name: Upload Harper logs
130-
if: failure()
131-
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
132-
with:
133-
name: harper-integration-api-test-logs-node-${{ matrix.node-version }}
134-
path: /tmp/hdb/log/
135-
retention-days: 3
136-
if-no-files-found: ignore
137-
138-
run-integration-apiTests-bun:
139-
name: Integration API Tests (Bun)
140-
if: false
141-
needs: [build]
142-
runs-on: ubuntu-latest
143-
timeout-minutes: 15
144-
145-
steps:
146-
- name: Checkout code
147-
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
148-
149-
- name: Setup Node.js
150-
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
151-
with:
152-
node-version: 24
153-
package-manager-cache: false
154-
155-
- name: Setup Bun
156-
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2
157-
158-
- name: Download build artifacts
159-
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
160-
with:
161-
name: harper-build-artifacts-node-24
162-
163-
- name: Setup Harper
164-
env:
165-
DEFAULTS_MODE: 'dev'
166-
HDB_ADMIN_USERNAME: 'admin'
167-
HDB_ADMIN_PASSWORD: 'password'
168-
ROOTPATH: '/tmp/hdb'
169-
OPERATIONSAPI_NETWORK_PORT: 9925
170-
LOGGING_LEVEL: 'debug'
171-
LOGGING_STDSTREAMS: true
172-
THREADS_COUNT: 1
173-
THREADS_DEBUG: false
174-
NODE_HOSTNAME: 'localhost'
175-
run: |
176-
mkdir -p /tmp/hdb/log
177-
node ./dist/bin/harper.js install > /tmp/hdb/log/install-stdout.log 2> /tmp/hdb/log/install-stderr.log
178-
sleep 10
179-
bun ./dist/bin/harper.js start > /tmp/hdb/log/start-stdout.log 2> /tmp/hdb/log/start-stderr.log &
180-
sleep 10
181-
182-
- name: Run API Tests
183-
id: run-api-tests
184-
env:
185-
HDB_ADMIN_USERNAME: 'admin'
186-
HDB_ADMIN_PASSWORD: 'password'
187-
run: npm run test:integration:api-tests
188-
189-
- name: Upload Harper logs
190-
if: failure()
191-
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
192-
with:
193-
name: harper-integration-api-test-logs-bun
194-
path: /tmp/hdb/log/
195-
retention-days: 3
196-
if-no-files-found: ignore
197-
198-
run-integration-apiTests-windows:
199-
name: Integration API Tests (Windows, Node.js v24)
200-
needs: [build-windows]
201-
runs-on: windows-latest
202-
timeout-minutes: 15
203-
204-
steps:
205-
- name: Checkout code
206-
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
207-
208-
- name: Setup Node.js 24
209-
uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6.4.0
210-
with:
211-
node-version: 24
212-
package-manager-cache: false
213-
214-
- name: Download build artifacts
215-
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
216-
with:
217-
name: harper-build-artifacts-windows
218-
219-
- name: Setup Harper
220-
env:
221-
DEFAULTS_MODE: 'dev'
222-
HDB_ADMIN_USERNAME: 'admin'
223-
HDB_ADMIN_PASSWORD: 'password'
224-
ROOTPATH: ${{ runner.temp }}/hdb
225-
OPERATIONSAPI_NETWORK_PORT: 9925
226-
LOGGING_LEVEL: 'debug'
227-
LOGGING_STDSTREAMS: true
228-
THREADS_COUNT: 1
229-
THREADS_DEBUG: false
230-
NODE_HOSTNAME: 'localhost'
231-
run: |
232-
New-Item -ItemType Directory -Force -Path ${{ runner.temp }}/hdb/log
233-
node ./dist/bin/harper.js install > ${{ runner.temp }}/hdb/log/install-stdout.log 2> ${{ runner.temp }}/hdb/log/install-stderr.log
234-
Start-Process node -ArgumentList "./dist/bin/harper.js start" -RedirectStandardOutput "${{ runner.temp }}/hdb/log/start-stdout.log" -RedirectStandardError "${{ runner.temp }}/hdb/log/start-stderr.log" -PassThru
235-
Write-Host "Waiting for Harper to be ready on port 9925..."
236-
$deadline = (Get-Date).AddSeconds(120)
237-
do {
238-
Start-Sleep -Milliseconds 500
239-
$client = New-Object System.Net.Sockets.TcpClient
240-
$connected = $client.ConnectAsync('localhost', 9925).Wait(2000)
241-
if ($connected) { $client.Close(); $ready = $true }
242-
else { $client.Close() }
243-
} until ($ready -or (Get-Date) -gt $deadline)
244-
if (-not $ready) { Write-Error "Harper did not start within 120s"; exit 1 }
245-
Write-Host "Harper is ready."
246-
247-
- name: Run API Tests
248-
id: run-api-tests
249-
env:
250-
HDB_ADMIN_USERNAME: 'admin'
251-
HDB_ADMIN_PASSWORD: 'password'
252-
run: npm run test:integration:api-tests
253-
254-
- name: Upload Harper logs
255-
if: failure()
256-
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
257-
with:
258-
name: harper-integration-api-test-logs-windows
259-
path: ${{ runner.temp }}/hdb/log/
260-
retention-days: 3
261-
if-no-files-found: ignore
262-
26377
run-integration-tests:
26478
name: Integration Tests ${{matrix.shard}}/4 (Node.js v${{ matrix.node-version }})
26579
runs-on: ubuntu-latest

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@ harper-*.tgz
5050
# AI
5151
.antigravitycli/
5252
.claude/
53+
cache/
5354

5455
# YCSB benchmark results (generated)
5556
benchmarks/ycsb/results/

‎AGENTS.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,8 @@ npx mocha unitTests/resources/mytest.js
4343

4444
TypeScript is stripped at runtime via `--conditions=typestrip` (Node.js native type stripping) — no compilation required for development. Use `npm run test:unit:typestrip` to run tests with this mode.
4545

46+
**Test timing:** prefer condition-waits over fixed `delay(N)` sleeps. `await delay(N); assert(sideEffectHappened)` races against loaded runners and is the root cause of a class of flakiness (#1138). Use the shared `waitFor(condition, timeout?, interval?)` helper in `unitTests/waitFor.js` to poll until the actual condition holds. Reserve fixed sleeps for genuinely modeling elapsed time (TTL/expiry windows) or asserting a non-event (that something has _not_ happened yet).
47+
4648
---
4749

4850
## Architecture

‎DESIGN.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,9 @@ Adding a new system table (e.g. `hdb_deployment` in #641 Slice A) requires three
100100

101101
1. **`json/systemSchema.json`** — the table entry. Fresh installs auto-create it via `utility/mount_hdb.ts:createTables()`, which iterates `Object.keys(systemSchema)` on first boot.
102102
2. **`utility/hdbTerms.ts`** — add the table name to `SYSTEM_TABLE_NAMES`.
103-
3. **`upgrade/directives/<version>.ts`** — provisions the table on existing installs that already have a system schema. Registered in `upgrade/directives/directivesController.ts` (which is otherwise empty — its `versions` Map gets populated by these imports). The directive shape is `{ version, sync_functions, async_functions }`; copy `5-2-0.ts` for the canonical pattern (uses `bridge.createTable` to match what `mount_hdb` does on a fresh install).
103+
3. **`upgrade/directives/<version>.ts`** — provisions the table on existing installs that already have a system schema. Registered in `upgrade/directives/directivesController.ts` (which is otherwise empty — its `versions` Map gets populated by these imports). The directive shape is `{ version, sync_functions, async_functions }`; copy `5-1-0.ts` for the canonical pattern (uses `bridge.createTable` to match what `mount_hdb` does on a fresh install).
104+
105+
**Version the directive to the first release that ships the dependent code, not a later one.** Directives only run when `current_version < directive_version <= upgrade_version` (`directivesController.getVersionsForUpgrade`). The `hdb_deployment` directive was originally mis-tagged `5.2.0` while the deployment-recorder code shipped in `5.1.0`, so on every `5.0.x -> 5.1.x` upgrade the directive was filtered out (`5.2.0 > 5.1.x`) and the table never got created — breaking replicated `deploy_component` on peer nodes for the entire existing customer base. Caveat: `utility/common_utils.ts:compareVersions` strips trailing `.0` and therefore sorts a pre-release (`5.1.0-beta.1`) _above_ its GA (`5.1.0`), so an install already on a `5.1.0-beta.x` data version will not pick up a `5.1.0` directive when upgrading to GA; those pre-release installs need the table created by other means.
104106

105107
System tables replicate by default. To opt out, add the name to `NON_REPLICATING_SYSTEM_TABLES` in `resources/databases.ts`. The check happens after table init and sets `table.replicate = false` per-node.
106108

0 commit comments

Comments
 (0)