Skip to content

Commit 0b5d908

Browse files
authored
docs(onboarding): pytz install, render flags, sample-6B auto-approve, headless sign-off (ACE-063) (#125)
* docs(onboarding): pytz install, render flags, sample-6B auto-approve, headless sign-off Docs/packaging half of the OMOP-onboarding fixes (pairs with ACE-062 code): - B1: duckdb install now 'duckdb pytz' in both driver tables + why (timestamptz needs pytz) - B2: 6B render_model_explorer invocation uses explicit --profile/--artifacts-dir/--out (no implied --root) - B3: 6B silent build auto-approves the pre-seed queue as signer=system before seeding, so it clears its own preseed gate (real DBs keep the human gate) - B4: agami-model documents the no-browser 'sm approve-queue' sign-off path; clarifies PII/should_open_explorer is advisory, not a seed blocker Spec: ACE-063 * docs: correct review-queue output shape to counts.total (Copilot review on #125) sm review-queue nests the count under counts.total, not a top-level total. Spec: ACE-063
1 parent ff3985b commit 0b5d908

3 files changed

Lines changed: 37 additions & 3 deletions

File tree

plugins/agami/shared/connection-reference.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -224,10 +224,12 @@ The "Python Driver Fallback" section further down shows the inline `python3 -c '
224224
| MySQL / MariaDB | 3306 | `mysql` | `pymysql` (`pip install pymysql`) | optional |
225225
| **Snowflake** | 443 (HTTPS) | `snowsql` | `snowflake-connector-python` (`pip install snowflake-connector-python`) | TLS always (managed by client) |
226226
| SQLite | N/A (file) | `sqlite3` | built-in `sqlite3` | n/a |
227-
| DuckDB | N/A (file) | `duckdb` | built-in or `pip install duckdb` | n/a |
227+
| DuckDB | N/A (file) | `duckdb` | built-in or `pip install duckdb pytz` | n/a |
228228

229229
Supported end-to-end: Postgres + Redshift + MySQL + Snowflake + SQLite + **BigQuery**. SQLite also works via DuckDB. Other databases (SQL Server, Oracle, Databricks, ClickHouse) are deferred.
230230

231+
> **Why `pytz` with DuckDB:** the DuckDB Python driver needs `pytz` to materialize `TIMESTAMP WITH TIME ZONE` values into Python — without it, any query selecting a timestamptz column fails at runtime (`Required module 'pytz' failed to import`). Install it alongside `duckdb`.
232+
231233
---
232234

233235
## CLI Connection Commands

plugins/agami/skills/agami-connect/SKILL.md

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -227,9 +227,11 @@ Native CLIs (optional fast path for *queries* — introspection doesn't use them
227227
| oracle | `import oracledb` | `oracledb` |
228228
| databricks | `from databricks import sql` | `databricks-sql-connector` |
229229
| trino | `import trino` | `trino` |
230-
| duckdb | `import duckdb` | `duckdb` |
230+
| duckdb | `import duckdb, pytz` | `duckdb pytz` |
231231
| sqlite | stdlib — always present ||
232232

233+
> **duckdb needs `pytz`** to materialize `TIMESTAMP WITH TIME ZONE` values into Python — without it a query that selects a timestamptz column fails at runtime (`Required module 'pytz' failed to import`), so it's part of the driver install, not optional.
234+
233235
If the driver is missing, **confirm via AskUserQuestion**, then `"$PY" -m pip install --user <package>` (plain `pip install` fallback). Same "never install silently" convention as the model deps (0a.5b). Do this for `$PY` so `sm introspect` connects on the first try.
234236

235237
### 0a.5b — Ensure the semantic-model dependencies
@@ -372,7 +374,18 @@ If `<artifacts_dir>/agami-example/org.yaml` already exists, the sample is alread
372374
- **6A (copy, < 1 min):** `mkdir -p "<artifacts_dir>/agami-example"` then `cp -R "$AGAMI_PLUGIN_ROOT/samples/store/model/." "<artifacts_dir>/agami-example/"`. **Validate it loads here**: `bash "$AGAMI_PLUGIN_ROOT/scripts/sm" validate "<artifacts_dir>/agami-example"`. If it fails, surface the errors and stop — never leave a half-wired profile. Then **stamp a model_version** (a *copy* doesn't go through introspect/curate, so nothing auto-stamps it): `bash "$AGAMI_PLUGIN_ROOT/scripts/sm" snapshot "<artifacts_dir>/agami-example"` — best-effort, so the answer receipt shows a version rather than `null`. (We stamp at copy time instead of committing a static `.snapshots/` so it always matches the model's actual content; 6B gets one automatically from introspect.)
373375
- **6B (rebuild live — "watch it build"):** ignore the committed `model/` and run the **normal Phases 1→2** against the `agami-example` profile (`--db-type sqlite`) — the same introspect → enrich → seed pipeline a real onboarding uses, just pointed at the sample SQLite file. It takes a few minutes (the non-default option). **Don't mention tokens, cost, or billing** — surface time (~5–10 min), not scary money words.
374376
- **Sample carve-outs — the dataset is small + curated, so DON'T prompt (build silently over ALL tables):** skip the [Phase 1.6](#16--discover--prune-the-table-list-cheap-first-pass) **prune** page, skip the **org-description** prompt (Phase 2f / 0a), and skip the **doc/metrics intake** (Phase 1's "do you have a data dictionary / dbt repo?"). These prompts exist for a real unknown DB; for the sample they're noise. Introspect + enrich every sample table without asking.
375-
- **When the model validates, OPEN THE MODEL-EXPLORER so the user sees what was built** — render it in **browse mode** (`render_model_explorer.py` for `<artifacts_dir>/agami-example`, i.e. `/agami-model` browse — **not** the `/agami-model preseed` sign-off gate that ends the turn elsewhere in this skill). This is the whole point of "watch it build"; a prose-only wrap would defeat it. Render it *together with* step 7's short dataset description + starter questions (so don't cede the turn), so the user can both look at the model and start asking. (Do **not** render the NL→SQL examples-validation page — lower-value for the curated sample.)
377+
- **Clear the pre-seed gate before seeding (the silent build has no human to sign off).** Enrichment lands the sample's metrics/entities `unreviewed`, so `seed-examples` would refuse with `preseed_review_pending`. Because the sample is curated and trusted, **auto-approve the queue as a system signer** right before seeding — the silent path clears its own gate:
378+
```bash
379+
bash "$AGAMI_PLUGIN_ROOT/scripts/sm" approve-queue "<artifacts_dir>/agami-example" --signer system --role system
380+
```
381+
(This is only sanctioned for the curated sample; a real database keeps the human sign-off gate.)
382+
- **When the model validates, OPEN THE MODEL-EXPLORER so the user sees what was built** — render it in **browse mode** (i.e. `/agami-model` browse — **not** the `/agami-model preseed` sign-off gate that ends the turn elsewhere in this skill). The script takes the profile and artifacts dir as **separate** flags (there is no `--root`); use the same invocation the [`/agami-model` skill documents](../agami-model/SKILL.md):
383+
```bash
384+
python3 "$AGAMI_PLUGIN_ROOT/scripts/render_model_explorer.py" \
385+
--profile agami-example --artifacts-dir "<artifacts_dir>" \
386+
--out "<artifacts_dir>/local/model/agami-example/<ts>.html"
387+
```
388+
This is the whole point of "watch it build"; a prose-only wrap would defeat it. Render it *together with* step 7's short dataset description + starter questions (so don't cede the turn), so the user can both look at the model and start asking. (Do **not** render the NL→SQL examples-validation page — lower-value for the curated sample.)
376389

377390
**6A****step 7** (describe + stop). **6B** → step 7's description + starter questions **and** open `/agami-model`. Both end with a validated `<artifacts_dir>/agami-example/` model.
378391
7. **Wrap up — describe the dataset, offer questions, then STOP. Do NOT auto-run a query.** This is the entire closing for the sample path: **skip the rest of Phases 3–8** (no introspect summary, no "re-introspect `<profile>`" / "when you want the real thing" framing — that pushes the user off the sample they just picked and can surface another profile's name). The user asked to *query* the sample, not watch a scripted demo — so hand them the keys, don't drive. **(Exception: 6B already opened `/agami-model` — that's the one review surface the "watch it build" path keeps; see 6B. The 6A copy path opens nothing.)**
@@ -769,6 +782,8 @@ bash "$AGAMI_PLUGIN_ROOT/scripts/sm" curate-gate "$ROOT"
769782
```
770783
`{pii_count, preseed_count, should_open_explorer}`. **PII count** = columns flagged `sensitive` still queryable (an excluded column, or any column under an excluded table, isn't counted — so once the user excludes them it drops to 0 and the gate stops re-opening). **preseed count** = metrics + named-filters + entities needing sign-off (relationships are NOT gated — FK joins are engine-approved, inferred joins self-approve as you query).
771784
785+
> **PII is advisory, not a seed blocker.** `should_open_explorer` can be true purely from a non-zero `pii_count` (sensitive columns still queryable) — that's a *review nudge* (the user's call to exclude or keep), **not** a gate on seed generation. Only a non-zero `preseed_count` (unreviewed metrics/entities) actually blocks `seed-examples`. Don't tell the user PII is stopping their seeds; it isn't.
786+
772787
**If `should_open_explorer` is true → invoke `/agami-model preseed` and END THE TURN.** The explorer is the **single** curation surface, with task-focused tabs so nothing is buried:
773788
- the **PII tab** — every flagged column *and* every suspected-but-unflagged one (e.g. `first_name` in `sys_user`) in one list, each with a confirm/clear toggle. This is where the user reviews PII without hunting through tables.
774789
- the **Metrics tab** — the proposed measures grouped by table and collapsed (`incident · 9 [✓ Approve 9]`), with per-table and "approve all proposed" bulk buttons, so a few-hundred-metric set signs off fast.

plugins/agami/skills/agami-model/SKILL.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,6 +220,23 @@ Then end the turn. The skill is one-shot per invocation — re-enter via the sla
220220

221221
---
222222

223+
## Sign-off without a browser (`sm approve-queue`)
224+
225+
This dashboard is HTML — it assumes a browser. On a **headless machine** (no browser, e.g. a remote/SSH-only host), sign off the whole pending queue from the CLI instead. `sm approve-queue` reads the same queue this dashboard's Review tab shows (Rule 1 metrics/named-filters + Rule 2 joins/entities), stamps each item, and applies it in one call:
226+
227+
```bash
228+
ROOT="<artifacts_dir>/<profile>"
229+
bash "$AGAMI_PLUGIN_ROOT/scripts/sm" approve-queue "$ROOT" --signer you@example.com --role owner
230+
# --kind metric|entity|relationship → narrow to one type (repeatable)
231+
# --dry-run → print the approve ops without applying
232+
```
233+
234+
- **`--signer` and `--role` are required** — an approve must record *who* signed off (the validator rejects an approved entry with no sign-off stamp). The command self-stamps the `at` timestamp, so you don't build ops by hand.
235+
- Afterward `sm review-queue "$ROOT"` shows `counts.total: 0` and `sm curate-gate "$ROOT"` drops `preseed_count` to 0.
236+
- **PII does not block seeding.** `curate-gate` may still report `should_open_explorer: true` purely because sensitive columns remain queryable — that's **advisory** (your call to exclude or keep them), **not** a gate on seed generation. Only unreviewed **pre-seed** items (metrics/entities) block `seed-examples`; approving the queue clears that.
237+
238+
---
239+
223240
## What the runtime does with `rejected` entries
224241

225242
The trust spine has always read `agami.review_state`. The model loader in [`plugins/agami/skills/agami-query/SKILL.md → Phase 1c`](../agami-query/SKILL.md#1c--index-the-model-for-fast-access) filters entries with `review_state: rejected` out of `datasets_by_name`, `datasets_by_qname`, `fields_by_qname`, and `relationships_by_endpoints`. Rejected entries:

0 commit comments

Comments
 (0)