|
| 1 | +--- |
| 2 | +name: synthscholar-skills |
| 3 | +version: 0.1.0 |
| 4 | +description: Set up and query SynthScholar / PRISMA systematic reviews. (1) Guided protocol intake — walk a user through EVERY required and optional input (research question, PICO, inclusion/exclusion criteria, databases, risk-of-bias tool, charting questions, per-group/cohort analysis, appraisal domains, output formatting, PRISMA registration) and validate completeness before a run. (2) Provenance queries — list included publications, distinguish full-text vs abstract-only, retrieve full-text content, and audit where each full text came from, across both the RDF/SLR-ontology export (SPARQL) and the PostgreSQL article store (SQL). Use when asked to start/scope a review, gather review inputs, check a protocol is complete, list included-with-full-text vs abstract-only, retrieve full-text content, or audit full-text provenance. |
| 5 | +license: Apache-2.0 |
| 6 | +--- |
| 7 | + |
| 8 | +# SynthScholar Review Skills |
| 9 | + |
| 10 | +Two capabilities for working with systematic reviews produced by SynthScholar: |
| 11 | + |
| 12 | +1. **Guided protocol intake** — before a review runs, guide the user through |
| 13 | + *every* input the agent supports and validate the protocol is complete. |
| 14 | +2. **Provenance queries** — after a review runs, answer *which included |
| 15 | + publications have full text, which are abstract-only, what the content is, |
| 16 | + and where it came from*. |
| 17 | + |
| 18 | +Pick the mode from the request: setting up / scoping a review → intake; |
| 19 | +inspecting a finished review → queries. |
| 20 | + |
| 21 | +--- |
| 22 | + |
| 23 | +## Mode 1 — Guided protocol intake |
| 24 | + |
| 25 | +The `ReviewProtocol` has ~40 fields. **Do not interrogate the user field by |
| 26 | +field** — that is the wrong interaction. A user often gives only a title |
| 27 | +("a systematic review of vocal biomarkers for depression"). From that, you can |
| 28 | +draft a *complete, sensible* protocol yourself. The intake's job is to **draft, |
| 29 | +lock the few decisions that genuinely change the outcome, and confirm** — not |
| 30 | +to collect 40 answers. |
| 31 | + |
| 32 | +### The interaction pattern (do this) |
| 33 | + |
| 34 | +1. **Draft the whole protocol from whatever the user gave you.** Infer objective, |
| 35 | + full PICO, inclusion/exclusion criteria, databases, and domain features |
| 36 | + (charting questions, `grouping_dimension`, appraisal domains) with |
| 37 | + domain-appropriate defaults. Use |
| 38 | + [references/protocol_intake.md](references/protocol_intake.md) as the field |
| 39 | + catalog — every field's default and validation is there. |
| 40 | + |
| 41 | +2. **Identify the 2–4 *pivotal* decisions** — the ones where a wrong default is |
| 42 | + costly to run on, and ask **only those**, as structured questions with a |
| 43 | + **recommended** option pre-selected. Everything else stays a silent default. |
| 44 | + The usual pivots (not fixed — judge per topic): |
| 45 | + - **Scope of the key concept** — how broadly to read the intervention/exposure |
| 46 | + for inclusion (narrow vs. include adjacent methods). Drives recall. |
| 47 | + - **Appraisal instrument** (`rob_tool`) — the default `RoB 2` assumes RCTs; |
| 48 | + most observational/diagnostic corpora need `ROBINS-I`, `QUADAS-2`, etc. |
| 49 | + - **Publication date window** — unbounded vs. a cutoff. |
| 50 | + - Occasionally: primary **outcome** framing, or the **cohort/grouping** |
| 51 | + dimension for per-group analysis. |
| 52 | + |
| 53 | + Surface these through the host's structured-question UI — **`AskUserQuestion` |
| 54 | + in Claude Code**, or the decision-card UI in the web app — one question each, |
| 55 | + each with a recommended answer and 2–4 concrete options. Ask the pivots |
| 56 | + together in one batch, not one message at a time. |
| 57 | + |
| 58 | +3. **Show the full drafted protocol back** (all fields, not just the answered |
| 59 | + ones) and get a single confirm before running. Make clear which values were |
| 60 | + the user's explicit answers vs. your defaults. |
| 61 | + |
| 62 | +4. **Record provenance** — each *asked* decision becomes a `slr:UserInput` |
| 63 | + (`question_asked`, `input_value`, `options_presented`) inside a |
| 64 | + `slr:PreWorkflowSession`; defaults you filled are attributed to the agent. |
| 65 | + See [references/data_model.md](references/data_model.md). |
| 66 | + |
| 67 | +5. **Validate before running:** |
| 68 | + |
| 69 | + ```bash |
| 70 | + python scripts/validate_protocol.py --print-template > protocol.json # scaffold |
| 71 | + # …write the drafted + confirmed values in… |
| 72 | + python scripts/validate_protocol.py protocol.json # gate |
| 73 | + ``` |
| 74 | + |
| 75 | + Exits non-zero if a required field is missing; warns on missing recommended |
| 76 | + fields / invalid enums. Only run the review once it passes. |
| 77 | + |
| 78 | +### Anti-patterns (don't do these) |
| 79 | + |
| 80 | +- ❌ Asking the user to fill every field, or pasting the whole checklist at them. |
| 81 | +- ❌ Asking about advanced features (batch size, cache TTL, section formats) |
| 82 | + unless the user raised them — default silently. |
| 83 | +- ❌ Running without showing the drafted protocol and getting a confirm. |
| 84 | +- ❌ Free-text-prompting the pivotal decisions when the host has a structured |
| 85 | + question UI — use the cards so the recommended option is one click. |
| 86 | + |
| 87 | +### Worked example |
| 88 | + |
| 89 | +User: *"a systematic review of vocal biomarkers for depression."* |
| 90 | + |
| 91 | +You draft the full protocol (objective, PICO, inclusion/exclusion, databases, |
| 92 | +charting questions, `grouping_dimension`, etc.) from that one line, then ask |
| 93 | +**only** the pivots as structured cards, each with a recommended answer: |
| 94 | + |
| 95 | +- **Scope of "vocal biomarkers" for inclusion?** → *Any speech-derived signal |
| 96 | + (incl. ASR & deep embeddings)* / Acoustic-prosodic features only / Clinician-rated only |
| 97 | +- **Appraisal instrument?** (RoB 2 assumes RCTs; this corpus is |
| 98 | + diagnostic/observational) → *ROBINS-I* / QUADAS-2 / Newcastle-Ottawa |
| 99 | +- **Publication date window?** → *Unbounded* / 2015+ / last 10 years |
| 100 | + |
| 101 | +Then show the complete drafted protocol and get one confirm before running. |
| 102 | +The three answers are recorded as `slr:UserInput`; every defaulted field is |
| 103 | +attributed to the agent. |
| 104 | + |
| 105 | +--- |
| 106 | + |
| 107 | +## Mode 2 — Provenance queries |
| 108 | + |
| 109 | +> *Which included publications have full text, which are abstract-only, what is |
| 110 | +> the full-text content, and where did it come from?* |
| 111 | +
|
| 112 | +The same review is materialised in **two places**, and this mode covers both: |
| 113 | + |
| 114 | +| Representation | Where | Query language | Full-text signal | |
| 115 | +| --- | --- | --- | --- | |
| 116 | +| RDF graph (SLR ontology) | exported `.ttl` / `.jsonld` | **SPARQL** | `slr:full_text_artifact` node | |
| 117 | +| PostgreSQL article store | `article_store` + `article_full_text` | **SQL** | row in `article_full_text` | |
| 118 | + |
| 119 | +Pick the surface that matches what the user has in hand. If they exported RDF, |
| 120 | +use SPARQL. If they have the live database, use SQL. The two are kept in |
| 121 | +parity: `slr:content_hash` in RDF equals `article_full_text.content_sha256`. |
| 122 | + |
| 123 | +## When to invoke this skill |
| 124 | + |
| 125 | +**Intake (Mode 1)** — when the user asks to: |
| 126 | + |
| 127 | +- **Start / scope a new review** or "set up a protocol". |
| 128 | +- **Gather the review inputs** — research question, PICO, inclusion/exclusion. |
| 129 | +- **Check a protocol is complete** before running. |
| 130 | +- Configure **domain features**: charting questions, per-group / cohort |
| 131 | + analysis, custom appraisal domains, output formatting, PRISMA registration. |
| 132 | + |
| 133 | +**Queries (Mode 2)** — when the user asks to: |
| 134 | + |
| 135 | +- **List included publications that have full text** (vs. everything included). |
| 136 | +- **Distinguish full-text-included from abstract-only** inclusions. |
| 137 | +- **Retrieve the full-text content** of included sources. |
| 138 | +- **Audit provenance** — which resolver produced each full text (`pmc_oa`, |
| 139 | + `europe_pmc_oa`, `unpaywall_pdf`, `biorxiv_pdf`, `openalex_pdf`, |
| 140 | + `semanticscholar_pdf`, `article_store`, `cache`) and when. |
| 141 | +- **Verify or backfill** the `article_full_text` provenance table. |
| 142 | + |
| 143 | +Mode 2 is scoped to full-text availability/content/provenance of *included* |
| 144 | +sources — not screening decisions or full charting extraction. |
| 145 | + |
| 146 | +## The core model |
| 147 | + |
| 148 | +An **included** publication is a `slr:IncludedSource` (SPARQL) / a row in |
| 149 | +`article_store` that survived screening (SQL). Among included publications: |
| 150 | + |
| 151 | +- **full-text-included** → full text was retrieved. In RDF it carries a |
| 152 | + `slr:full_text_artifact` → `slr:StoredArtifact` (kind `source_text_extract`) |
| 153 | + with `slr:content_text`, `slr:content_hash`, `slr:content_size_bytes`. In SQL |
| 154 | + it has a matching row in `article_full_text`. |
| 155 | +- **abstract-only** → included but no full text. It has `dcterms:abstract` |
| 156 | + (RDF) / a non-empty `abstract` in `article_store`, but **no** |
| 157 | + `slr:full_text_artifact` / **no** `article_full_text` row. |
| 158 | + |
| 159 | +Presence of the artifact / row **is** the distinguishing signal — never infer |
| 160 | +full-text availability from the length of an abstract. |
| 161 | + |
| 162 | +## How to answer a request |
| 163 | + |
| 164 | +1. **Determine the surface.** Ask (or infer from the artifacts present) |
| 165 | + whether the user has an RDF export (`.ttl`/`.jsonld`) or DB access (a DSN). |
| 166 | +2. **Pick the query.** Match the request to a named recipe in |
| 167 | + [references/sparql_queries.md](references/sparql_queries.md) or |
| 168 | + [references/sql_queries.md](references/sql_queries.md). |
| 169 | +3. **Run it** with the matching helper script, or hand the user the query. |
| 170 | +4. **Report** the distinction plainly: N included, of which M have full text |
| 171 | + and (N−M) are abstract-only; include provenance/source breakdown if asked. |
| 172 | + |
| 173 | +## Helper scripts |
| 174 | + |
| 175 | +All live in [scripts/](scripts/). Run them from an environment where the |
| 176 | +project's deps are importable (`rdflib` for SPARQL, `psycopg[binary]>=3.1` for |
| 177 | +SQL; the backfill also needs the `synthscholar` package on `PYTHONPATH`). |
| 178 | + |
| 179 | +```bash |
| 180 | +# Intake: blank template → fill from answers → validate before running |
| 181 | +python scripts/validate_protocol.py --print-template > protocol.json |
| 182 | +python scripts/validate_protocol.py protocol.json |
| 183 | + |
| 184 | +# SPARQL over an RDF export — named recipe or your own query |
| 185 | +python scripts/query_sparql.py review.ttl --query included-full-text |
| 186 | +python scripts/query_sparql.py review.ttl --query status-all --format csv |
| 187 | +python scripts/query_sparql.py review.ttl --sparql ./my_query.rq |
| 188 | + |
| 189 | +# SQL over the Postgres article store (DSN via --dsn or $PRISMA_PG_DSN) |
| 190 | +python scripts/query_postgres.py included-full-text |
| 191 | +python scripts/query_postgres.py source-breakdown |
| 192 | +python scripts/query_postgres.py get-content --pmid 39012345 |
| 193 | + |
| 194 | +# Backfill article_full_text from legacy article_store.full_text rows |
| 195 | +python scripts/backfill_full_text.py # needs migration 006 applied |
| 196 | +``` |
| 197 | + |
| 198 | +Run any script with `-h` for its full option list, and `--list` (query |
| 199 | +scripts) to see every named recipe. |
| 200 | + |
| 201 | +## References |
| 202 | + |
| 203 | +- [references/protocol_intake.md](references/protocol_intake.md) — complete |
| 204 | + guided-intake checklist: every `ReviewProtocol` input with question, example, |
| 205 | + validation, default, and tier. |
| 206 | +- [references/sparql_queries.md](references/sparql_queries.md) — SPARQL recipe catalog. |
| 207 | +- [references/sql_queries.md](references/sql_queries.md) — SQL recipe catalog. |
| 208 | +- [references/data_model.md](references/data_model.md) — the SLR-ontology terms |
| 209 | + and the `article_store` / `article_full_text` schema this skill relies on, |
| 210 | + plus how provenance is captured in the pipeline. |
| 211 | + |
| 212 | +## Caveats |
| 213 | + |
| 214 | +- Both signals are written **going forward**. Reviews exported/run before the |
| 215 | + full-text-provenance change won't have `slr:full_text_artifact` triples or |
| 216 | + `article_full_text` rows until re-exported or backfilled |
| 217 | + (`scripts/backfill_full_text.py`). |
| 218 | +- `full_text_source` is empty for backfilled historical rows — their provenance |
| 219 | + was never recorded. |
| 220 | +- Full-text bodies can be large; the SQL `get-content` recipe streams a single |
| 221 | + PMID. Avoid `SELECT content` across the whole table in interactive use. |
0 commit comments