Merge pull request #250 from JarvusInnovations/develop

Release: v2.4.0

Author: themightychris

CLAUDE.md

@@ -73,6 +73,45 @@ See [specs/architecture.md](specs/architecture.md) for the full stack rationale.
 - **Generated changes commit first.** When a command modifies files (`npm install`, codegen), commit those in a dedicated commit with the exact command in the body. Then make manual edits in a separate commit.
 - **Don't commit suspected secrets** — `.env`, anything in `*.local.*`, credentials, private keys.
 
+## Releases
+
+Two npm release tracks ship from this repo on **separate, prefix-namespaced git-tag
+tracks** — keep them distinct, and mind the ordering rule:
+
+- **`gitsheets`** (the JS package) — released on **`v*`** tags via the develop→main
+  Release-PR flow (`release-prepare`/`-validate`/`-publish` workflows; use the
+  **`release-flow`** skill). Merging the `Release: vX.Y.Z` PR cuts the tag;
+  `publish-npm.yml` then builds, tests, and publishes from that tag.
+- **`@gitsheets/core-napi`** (+ its 6 platform packages) — released on
+  **`core-napi-v*`** tags via `core-napi.yml`: native per-platform builds + npm
+  trusted publishing (OIDC). Never tag the addon with a bare `v*` — that namespace
+  belongs to the JS package.
+
+Rules learned the hard way (v2.3.0's failed publish — see
+`plans/core-napi-0.2.0-release.md`):
+
+1. **A `gitsheets-core` behavior change requires a core-napi release BEFORE the JS
+   release.** `publish-npm` runs the workspace napi tests against the *published*
+   platform binaries (resolved via the workspace manifest's `optionalDependencies`),
+   so a JS release atop stale binaries fails mid-publish with test assertions
+   mismatching old core behavior. If `rust/gitsheets-core` changed since the last
+   `core-napi-v*` tag, ship core-napi first.
+2. **napi versions are tag-stamped, never pre-committed.** `core-napi.yml` runs
+   `npm version $VERSION` from the tag name at publish time and errors ("Version not
+   changed") if the manifest already carries that version. Tag `develop` with the
+   **un-bumped** manifests; don't bump `rust/gitsheets-napi/package.json` (or
+   `npm/*/package.json`) ahead of the tag.
+3. **Sync manifests AFTER the core-napi publish.** Once the new version is live, PR
+   the sync: `rust/gitsheets-napi` version + platform `optionalDependencies` +
+   `npm/*/package.json` to the released version, `packages/gitsheets` dep range,
+   and a lockfile refresh. Without this, workspace tests keep resolving the previous
+   platform binaries. (Committing the just-released version is safe — the stamp step
+   only fails when the committed version equals the tag being stamped.)
+
+Sequence for a batch that touches core: merge the feature PRs → tag `core-napi-v*`
+on develop → wait for the 7 packages to land on npm → PR the manifest/lockfile sync →
+merge → run the release-flow skill on the auto-opened `Release: v*` PR.
+
 ## Tooling
 
 - **`gh-axi`** (or `gh`) for all GitHub operations.

README.md

@@ -3,24 +3,134 @@
 A git-backed document store for low-volume, high-touch, human-scale data.
 
 Records are TOML (or markdown-with-frontmatter) files in a git repository, laid
-out by a per-sheet path template. Every mutation is a commit, so the log is the
-audit trail. The engine is a shared **Rust core** with thin per-language
-bindings: this repo ships the Node binding (`gitsheets` on npm) and a Python
-binding, both running on the same core, so a write from either language produces
-byte-identical commits.
+out by a per-sheet path template. Every write is validated against your schema,
+serialized to a canonical form (the same data always produces the same bytes),
+and lands as a commit. The log is your audit trail, the diff is your review
+tool, and a `git clone` is a complete copy of everything. The engine is a
+shared Rust core with thin per-language bindings: the `gitsheets` npm package
+and a Python binding both produce byte-identical commits.
 
-The library and CLI live in [`packages/gitsheets`](packages/gitsheets); start
-there. The Rust core and its bindings live under [`rust/`](rust/).
+## The whole idea in one terminal
 
-## Spec-driven
+```console
+$ cat .gitsheets/contacts.toml
+[gitsheet]
+root = 'contacts'
+path = '${{ handle }}'
+
+[gitsheet.schema]
+type = 'object'
+required = ['handle', 'name']
+
+$ npx -y gitsheets upsert contacts '{"handle": "ada", "name": "Ada Lovelace"}'
+
+$ git show HEAD --stat --oneline
+f3a91c2 contacts upsert ada
+ contacts/ada.toml | 2 ++
+```
+
+A record is a file. A change is a commit. Everything git can do with commits
+(blame, revert, branch, push, diff, sign) now applies to your data.
+
+## What people run on it
+
+- **An application datastore.** [codeforphilly.org's rebuild](https://github.com/CodeForPhilly/codeforphilly-ng)
+  uses gitsheets as its only public datastore: about 32,000 person records,
+  where every mutating HTTP request becomes one commit authored as the acting
+  user. The audit system is `git log`.
+- **Operational databanks.** Inventories and worklists that used to rot in
+  spreadsheets, tracked as schema-validated records where every decision is a
+  reviewable commit.
+- **AI evaluation corpora.** Agents and models write verdicts as records
+  (keyed by subject and evaluator), one commit per pass, and nothing acts on a
+  verdict until a human has read the diff. The schema rejects malformed model
+  output at write time.
+- **Mirrors of external systems.** Extract an API into records, fix and merge
+  data in git where diffs and review are free, then load only the deltas back.
+
+## Getting started
+
+### The library
+
+```console
+npm install gitsheets        # Node >= 20; prebuilt engine, no Rust toolchain
+```
+
+```typescript
+import { openRepo } from 'gitsheets';
+
+const repo = await openRepo({ gitDir: 'data/.git' });
+
+await repo.transact(
+  { message: 'contacts upsert ada', author: { name: 'Ada', email: 'ada@example.org' } },
+  async (tx) => {
+    await tx.sheet('contacts').upsert({ handle: 'ada', name: 'Ada Lovelace' });
+  },
+);
+```
+
+One handler, one commit, committed only on success. `openStore` layers typed
+sheets on top with Standard Schema validators (Zod, Valibot, ArkType), and a
+successful transaction auto-refreshes reads. See
+[the docs](https://jarvusinnovations.github.io/gitsheets/) for the API guide.
+
+### The CLI
 
-[`specs/`](specs/) is the source of truth. Start at [`specs/README.md`](specs/README.md).
+Works in any repo with a `.gitsheets/` config, no install required:
+
+```console
+npx -y gitsheets query contacts --filter name='Ada Lovelace'
+npx -y gitsheets read contacts ada
+npx -y gitsheets normalize contacts
+```
+
+Installed globally it also mounts as a git subcommand: `git sheet query …`.
+
+### For AI agents: gitsheets-axi
+
+`gitsheets-axi` is the same store behind a CLI designed for agents: compact
+token-efficient output, schemas surfaced with every listing, and bulk
+operations (upsert, patch, delete) that land as single reviewable commits.
+
+```console
+npx -y gitsheets-axi sheets            # what's here, with schemas
+npx -y gitsheets-axi query contacts --group-by name
+jq -c '.[]' import.json | npx -y gitsheets-axi upsert contacts   # one commit
+```
+
+An agent working in a gitsheets repo can be given real write access: the
+schema gates every write and the history makes every action reversible.
+
+### The agent skill
+
+For Claude Code and compatible harnesses, an installable skill teaches the
+agent the config grammar, the API, and the axi tool, with session hooks that
+surface your sheets at startup:
+
+```console
+npx skills add JarvusInnovations/gitsheets -y --skill gitsheets
+```
+
+### Python
+
+A Python binding ([`rust/gitsheets-py`](rust/gitsheets-py), pyo3) runs on the
+same core — a write from Python and a write from Node produce byte-identical
+commits, proven by cross-binding tests in CI. It builds from the repo today;
+PyPI publication is on the roadmap.
+
+## What it is not
+
+Every write is a commit, so sustained high write rates are the wrong fit. One
+process writes to a repo at a time. Queries are index-assisted scans with
+filters, grouping, and counts, not SQL joins. Large media belongs in object
+storage, not records. If a conventional database is serving you well, keep it.
+
+## Spec-driven
 
-- [`specs/architecture.md`](specs/architecture.md) — stack and packaging
-- [`specs/rust-core.md`](specs/rust-core.md) — the Rust core + thin-binding architecture
-- [`specs/concepts.md`](specs/concepts.md) — Repository, Sheet, Record, Transaction, Store, Index
-- [`specs/api/`](specs/api/) — per-symbol API contracts
-- [`specs/behaviors/`](specs/behaviors/) — cross-cutting rules
+[`specs/`](specs/) is the source of truth for behavior; start at
+[`specs/README.md`](specs/README.md). The library and CLI live in
+[`packages/gitsheets`](packages/gitsheets); the Rust core and bindings live
+under [`rust/`](rust/).
 
 ## License
 

docs/path-templates.md

@@ -12,6 +12,7 @@ For the full spec see
 | --- | --- | --- |
 | Literal | `users/by-domain/` | Static path text |
 | Field reference | `${{ slug }}` | The value of `record.slug` |
+| Date bucket | `${{ publishedAt: YYYY/MM/DD }}` | UTC date partitions of the field — one directory level per format part |
 | Expression | `${{ slug.toLowerCase() }}` | Arbitrary JS expression with record fields in scope |
 | Recursive | `${{ path/** }}` | Field's value may contain `/` — for nested-path fields |
 | Prefix/suffix | `user-${{ id }}.draft` | Literal text attached to an expression |
@@ -26,13 +27,46 @@ path = "${{ slug }}"
 # Composite key (two-level directory)
 path = "${{ domain }}/${{ username }}"
 
-# Sharded by date
-path = "${{ publishedAt.getFullYear() }}/${{ publishedAt.getMonth() }}/${{ slug }}"
+# Date-bucketed (2026/03/09/my-post.toml)
+path = "${{ publishedAt: YYYY/MM/DD }}/${{ slug }}"
 
 # Nested path field
 path = "${{ contentPath/** }}"
 ```
 
+## Date buckets
+
+A date-bucket reference partitions records by a date field — by day, month,
+year, or ISO week:
+
+```toml
+path = "${{ publishedAt: YYYY/MM/DD }}/${{ slug }}"   # 2026/03/09/my-post.toml
+path = "${{ publishedAt: YYYY/MM }}/${{ slug }}"      # 2026/03/my-post.toml
+path = "${{ publishedAt: YYYY/WW }}/${{ slug }}"      # 2026/11/my-post.toml (ISO week)
+path = "${{ day: YYYY/MM/DD }}"                       # daily rollup: the bucket IS the key
+```
+
+The format is a **closed set** — `YYYY`, `YYYY/MM`, `YYYY/MM/DD`, `YYYY/WW`;
+anything else throws `ConfigError(config_invalid)` when the sheet is opened.
+Fancier partitioning falls back to the expression form.
+
+Semantics worth knowing:
+
+- **UTC always.** Buckets never consult the host timezone, so the same record
+  produces the same path on every machine. (The expression spelling
+  `getFullYear()` doesn't have this guarantee — it renders host-local dates.)
+- **`YYYY/WW` is ISO-8601 week numbering**, and its year part is the ISO
+  *week-based* year: 2027-01-01 belongs to ISO week 2026-W53 and renders as
+  `2026/53`.
+- **`MM`, `DD`, `WW` are always two digits** — partitions sort correctly.
+- The field may hold a datetime/date value or an ISO 8601 string. One bucket
+  token expands to multiple real directory levels, and must stand alone
+  between slashes.
+- Queries that supply the bucketed field (e.g.
+  `posts.queryAll({ publishedAt })`) descend the exact partition instead of
+  scanning every record; without it, the partition levels are walked like any
+  other unconstrained segment.
+
 ## How queries use the template
 
 When a query specifies fields the template uses, gitsheets prunes the tree

package-lock.json

@@ -41,7 +41,7 @@
   "license": "Apache-2.0",
   "author": "Jarvus Innovations",
   "dependencies": {
-    "@gitsheets/core-napi": "^0.2.0",
+    "@gitsheets/core-napi": "^0.3.0",
     "csv-parse": "^6.2.1",
     "csv-stringify": "^6.7.0",
     "rfc6902": "^5.2.0",