Feat/schema learning in process#377
Conversation
…arning Two-tier catalog: instance-wide schema_template (structure-only, dedup'd by hash) + per-project schema_catalog (values, counts, anomaly state) + schema_summary (materialized AI/query-editor doc). Pipeline lives in Pkg.SchemaLearning (Hot/Catalog/Worker/OpenApi); per-flush diffing in Worker.flushDirty replaces the legacy DB triggers. Drops apis.shapes / apis.fields / apis.formats / apis.facet_summaries and their anomaly triggers (migrations 0089 + 0090).
…ields/formats projectCacheById joined apis.shapes on every cache rebuild and filtered on sh.hash IS NOT NULL, so once 0090 dropped the table the cache for every project came back empty and the request path 500'd. Removes the join (and the now-unused shapeHashes field on ProjectCache) and strips the legacy shapes/fields/formats migration + delete steps from migrateAndDeleteMergedEndpoints — the schema-learning catalog re-derives structure per canonical key, so no row migration is needed.
projects.redacted_fields.field_category was dropped via CASCADE in 0090 (the apis.field_category enum it depended on was removed). The hot-path projectCache rebuild query still referenced it, throwing "column rf.field_category does not exist" on every ingestion batch and killing prod again. Replaces it with an empty middle segment so the '<>'-separated string format stays compatible with any downstream split.
Code ReviewWhat this PR doesFixes a broken SQL query in Issues1. Outdated struct/comment — The doc comment still says:
2. Overly verbose SQL comment The added SQL comment spans three lines explaining the migration context. Given the project's preference for conciseness, a single line is sufficient: -- field_category col dropped by 0090 CASCADE; middle segment is always empty now.3. concat(rf.endpoint_hash,'<>','<>', rf.path)
concat(rf.endpoint_hash, '<><>', rf.path)Both are equivalent (PostgreSQL 4. No test/CI coverage for the affected query The PR checklist has all items unchecked, and there's no test exercising Non-issues
|
Closes #
How to test
Checklist