|
| 1 | +# Changelog |
| 2 | + |
| 3 | +All notable changes to `structsense-skills`. Versions follow semantic versioning. |
| 4 | +The `version:` field in `SKILL.md` frontmatter is the source of truth; this file |
| 5 | +records what changed between versions so you can tell which features your local |
| 6 | +copy has. |
| 7 | + |
| 8 | +## [0.4.0] — 2026-06-05 (zero-hallucination policy) |
| 9 | + |
| 10 | +### Concept mapping policy (breaking) |
| 11 | +- **Mapping is mandatory and tool-only.** The cascade no longer silently |
| 12 | + skips alignment. Default cascade: local hybrid (`http://localhost:8000`, |
| 13 | + verified at `/docs`) → BioPortal → ask user for an alternate URL → |
| 14 | + **hard-stop** with an actionable error. The skill no longer falls back |
| 15 | + to OLS by default (OLS lacks gene coverage); pass `--allow-ols-fallback` |
| 16 | + to opt in. |
| 17 | +- **`concept_mapping_provenance: "llm_knowledge"` is forbidden in canonical |
| 18 | + output.** Any item carrying it is automatically demoted to `unmapped`, |
| 19 | + with `alignment_method: "validation_failed"`. The item itself is |
| 20 | + preserved (exhaustive extraction is not compromised); only the |
| 21 | + fabricated mapping is dropped. |
| 22 | + |
| 23 | +### New: `scripts/iri_validation.py` |
| 24 | +- Per-ontology regex patterns for UBERON, CL, NCBITaxon, MONDO, DOID, HP, |
| 25 | + CHEBI, DRON, BTO, OBI, MP, HGNC, NCBIGene, MGI, UniProt, PR, GO, EFO, |
| 26 | + NIFSTD, CIDO, plus a permissive structural fallback for unknown |
| 27 | + ontologies (`<NS>_<NUM>` OWL IRIs, identifiers.org, semanticweb.org). |
| 28 | +- Accepts **legitimate cross-ontology mappings** (e.g. CIDO results that |
| 29 | + reuse HP IRIs as synonyms) — the declared `ontology` field does NOT |
| 30 | + have to match the IRI prefix. We only reject `llm_knowledge` provenance |
| 31 | + and structurally-malformed IRIs. |
| 32 | +- Adds `result["validation"]` with `passed` / `demoted` counts and |
| 33 | + `demoted_by_reason` breakdown (`llm_knowledge_rejected`, |
| 34 | + `malformed_iri`, `wrong_ontology_pattern`, `missing_ontology_id`). |
| 35 | + |
| 36 | +### Stats clarity |
| 37 | +- **`stats.totals` block at the top** of every result. Top-level |
| 38 | + `total_items`, `total_entity_mentions`, `total_key_term_mentions`, |
| 39 | + `total_resources`, `unique_entities`, `unique_key_terms`. The human |
| 40 | + summary shows `TOTAL ITEMS: …` as the second line after `task_type`. |
| 41 | +- **`stats.task_type` auto-inferred** when missing — entities present → |
| 42 | + `ner`; resources → `resource`; activity/items → `structured_extraction`. |
| 43 | +- **`alignment.by_method`** no longer shows `{missing: N}` for items that |
| 44 | + have `concept_mapping_provenance` set. The normalizer fills |
| 45 | + `alignment_method` from the provenance (`tool` → `direct_tool_call`, |
| 46 | + `skipped` → `skipped`). |
| 47 | + |
| 48 | +### Connecting / runtime |
| 49 | +- **`connecting/claude-desktop.md`** added. Explains Claude Desktop's |
| 50 | + **split execution model** (chat UI on your machine, code interpreter in |
| 51 | + Anthropic's cloud sandbox), why your `localhost:8000` is unreachable |
| 52 | + from a sandboxed code-interpreter call (`/home/claude/work/...` paths |
| 53 | + are the giveaway), and how to bridge via an MCP server configured in |
| 54 | + `claude_desktop_config.json`. |
| 55 | + |
| 56 | +### Stats fix verified against real numbers |
| 57 | +Smoke-tested against the user's actual run (2331 entity mentions across |
| 58 | +15 labels, 210 key terms, 922 tool-mapped + 1619 skipped). The new |
| 59 | +output: |
| 60 | +- `totals.total_items: 2541` (prominent at top) |
| 61 | +- `task_type: "ner"` (was `null`) |
| 62 | +- `alignment.by_method: {direct_tool_call: 922, skipped: 1409, missing: 210}` (was `{missing: 2541}`) |
| 63 | + |
| 64 | +## [0.3.1] — 2026-06-05 (later same day) |
| 65 | + |
| 66 | +### Concept-mapping (fixed) |
| 67 | +- **API schema bug fixed.** `scripts/local_hybrid_map.py` was POSTing |
| 68 | + `{terms: [...]}` but the real `/map/batch` endpoint takes |
| 69 | + `{text: [{text, context?}], max_results}`. Verified against a live |
| 70 | + deployment. The client now supports either a flat list of strings or |
| 71 | + shaped dicts with optional per-term `context`. Also handles the real |
| 72 | + response shape (`results` is a dict keyed by input text, score is |
| 73 | + `final_score`). |
| 74 | +- **`prompts/alignment-via-http.md`** added: a turnkey curl + jq pipeline |
| 75 | + for hitting `/map/batch` directly. Use this when in Claude Code (Bash |
| 76 | + available) but the Python client isn't loaded. |
| 77 | +- **`SKILL.md` rule #10 strengthened**: "you MUST probe `curl |
| 78 | + ${MAPPER_URL}/docs` before declaring the mapper unavailable." Plus a |
| 79 | + runtime-reachability table (Claude Code → yes; claude.ai web → |
| 80 | + no, needs MCP bridge; ChatGPT cloud → no, needs Action). |
| 81 | + |
| 82 | +### New scripts ported from structsense |
| 83 | +- **`scripts/input_loader.py`** — PDF / CSV / TXT / MD ingestion. PDF chain: |
| 84 | + GROBID → PyMuPDF (`fitz`) → pdfminer.six. CLI writes a sibling `<stem>.txt` |
| 85 | + so subsequent NER stages have stable character offsets. |
| 86 | +- **`scripts/task_detection.py`** — auto-detect task type (ner / resource / |
| 87 | + structured_extraction / relation_extraction / keyphrase_extraction / …) |
| 88 | + from a free-text description. Heuristic regex first; LLM fallback via |
| 89 | + `llm_client`. |
| 90 | +- **`scripts/model_context.py`** — model context-window registry (~50 model |
| 91 | + families, longest-match wins) + `estimate_payload_tokens` + |
| 92 | + `compute_downstream_chunk_size` for sizing alignment/judge chunks |
| 93 | + programmatically. |
| 94 | +- **`group_by_entity.unify_ontology_across_entities`** — when the same |
| 95 | + surface form gets different ontology IDs from different chunks/models |
| 96 | + (parallel extraction is the usual cause), unify on the best one |
| 97 | + (tool-mapped > llm_knowledge > unmapped). Runs automatically inside |
| 98 | + `attach_grouped_views`. Mentions are preserved; only the ontology |
| 99 | + fields are normalized. |
| 100 | + |
| 101 | +## [0.3.0] — 2026-06-05 |
| 102 | + |
| 103 | +### Schema (breaking) |
| 104 | +- **`paper_title` / `doi` removed from per-entity items.** They now live ONCE at |
| 105 | + the top level under `source_metadata: { paper_title, doi, source_path }`. |
| 106 | + `paper_location` (section / page / paragraph) stays per-entity because it |
| 107 | + varies per mention. |
| 108 | +- **`source_model` provenance** added to every entity item (HF model id, or |
| 109 | + `llm_ner:<llm_model>` for the LLM extractor). |
| 110 | +- **`entities_grouped[]`** (and `key_terms_grouped[]`) added: per-entity index |
| 111 | + collapsing raw mentions by canonical `(entity.lower(), label)`. Each group has |
| 112 | + `source_models[]`, `source_model_counts{}`, `consensus_count`, |
| 113 | + `judge_score_max/avg/min`, deduplicated `sentences[]` with their |
| 114 | + `paper_locations[]`, and slim per-occurrence `mentions[]`. |
| 115 | +- **`stats` block** always embedded: totals, label histogram, |
| 116 | + **by_source_model** histogram, alignment provenance, judge score buckets, |
| 117 | + per-stage timings. |
| 118 | + |
| 119 | +### Pipeline |
| 120 | +- **Post-processing normalizer** (`scripts/normalize_result.py`) added. Runs |
| 121 | + automatically before every save, and is also exposed as a CLI so legacy |
| 122 | + result files can be normalized in place. It lifts per-entity |
| 123 | + `paper_title`/`doi` into `source_metadata`, tags missing `source_model`, |
| 124 | + attaches `entities_grouped`, and computes `stats`. **Idempotent.** |
| 125 | +- **Mapper cascade**: local hybrid (`http://localhost:8000`, verify via |
| 126 | + `/docs`) → BioPortal → ask the user for an alternative URL → skip alignment. |
| 127 | +- **HuggingFace NER ensemble** (`scripts/ner_models.py`): d4data, |
| 128 | + BC5CDR-chem, NCBI-disease, BioBERT-genetic, BENT-PubMedBERT family |
| 129 | + (Gene/Chemical/Disease/Anatomical/Cell-Type/Cell-Line/Organism/Bioprocess), |
| 130 | + Clinical-AI-Apollo, blaze999. Selectable via `--ner-profile`: |
| 131 | + `biomedical_broad` / `cns_cells` / `pharmacology` / `genetic` / `clinical` / |
| 132 | + `minimal` / `all`. |
| 133 | +- **Final filename** is `<input_stem>_final.json` by default. |
| 134 | + |
| 135 | +### NER prompts |
| 136 | +- Three variants: **general**, **neuroscience**, **CNS-cells**. |
| 137 | +- Each ships with explicit **EXHAUSTIVENESS** rules ("extract every occurrence, |
| 138 | + do not deduplicate by surface form") and explicit **WRONG / RIGHT** example |
| 139 | + blocks for the `source_metadata` shape. |
| 140 | +- **Mask-recall pass** (`prompts/mask-recall-pass.md`) and **mask-verify pass** |
| 141 | + (`prompts/mask-verify-pass.md`) for boosting recall and detecting label |
| 142 | + errors. |
| 143 | + |
| 144 | +### Connecting (new) |
| 145 | +- `connecting/claude-code.md` — install as a Claude Code skill (`~/.claude/skills/` |
| 146 | + or `.claude/skills/`). |
| 147 | +- `connecting/claude-skills.md` — publish as a hosted Anthropic Skill. |
| 148 | +- `connecting/custom-gpt.md` — wire as an OpenAI Custom GPT (Instructions + |
| 149 | + Knowledge files + Actions). |
| 150 | +- `connecting/mcp-server.md` — expose as an MCP server. |
| 151 | + |
| 152 | +## [0.2.0] — 2026-06-04 |
| 153 | + |
| 154 | +### Added |
| 155 | +- Multi-stage pipeline pattern (extractor → alignment → judge → human feedback). |
| 156 | +- Three NER variants and resource / structured-extraction prompts. |
| 157 | +- Ontology mapping clients: BioPortal, OLS, local hybrid. |
| 158 | +- JSON repair, span validator, chunking helpers. |
| 159 | +- Reference pipeline driver (`scripts/pipeline.py`). |
| 160 | + |
| 161 | +## [0.1.0] — 2026-06-03 |
| 162 | + |
| 163 | +- Initial skill scaffolding from the structsense pipeline patterns. |
0 commit comments