Commit 030b103
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?v=4&size=40){kind=avatar}
hasql-interpolate + record-dot migration: derived ENUM/CITEXT decoders (#409)
* wip: hasql-interpolate + record-dot migration in progress
* telemetry: derive FromRow/DecodeRow for OtelLogsAndSpans
Reorder OtelLogsAndSpans fields to match otelSpanColsSql so both row
instances drop to one-liner anyclass derives. Add the previously-absent
errors and message_size_bytes columns to the SELECT path (and errors to
the INSERT path) so they round-trip instead of being silently defaulted
to Nothing / 0 on read. errors switches to Maybe (AesonText Value) for
consistency with body/events. Consolidates spanRecordByName onto the
shared otelSpanColsSql snippet so it can't drift again. Net: ~110 lines
of hand-written instance plumbing gone.
* hasql: cast int4 columns to bigint in two SELECTs
QueryCache.hs (hit_count) and Issues.hs (affected_requests,
affected_clients) decode into Haskell Int (= int8) but the underlying
columns are INTEGER (int4), so every dashboard fetch and every cache
lookup was throwing UnexpectedColumnTypeStatementError under hasql.
Cast on the SELECT rather than narrowing the record field — safer
because the same field types are used as Int across the codebase.
* fix decode-type mismatches surfaced by hasql-interpolate strictness
apis.issues.seq_num is INT; cast to bigint in the issue-list SELECT so
the generic DecodeRow doesn't blow up on int4-vs-int8.
projects.query_library.query_type is an enum; cast to text in the three
UNION branches of queryLibHistoryForUser so the generic decoder reads it
as Text instead of a custom OID.
Add migration 0096 to convert the four remaining VARCHAR(n) columns
(users.first_name/last_name, projects.teams.name/handle) to TEXT.
VARCHAR's only practical difference is the catalog OID, and that OID
mismatch was crashing every team-list and member-list lookup under the
new hasql decoders.
* drop vendored stackage-pins.config; pin via cabal.project.freeze
cabal.project imported a 3300-line in-repo stackage snapshot just to strip
hs-opentelemetry-* pins so the source-repository-package override at 1.0.0.0
could win. cabal.project.freeze already covers all pinned versions
including source-repo packages, so the vendored snapshot is redundant.
Also fixes CI: the Dockerfile COPY glob did not include stackage-pins.config,
so docker builds were failing with 'stackage-pins.config: does not exist'.
* nav: keep API Catalog highlighted on /endpoints pages
The sidenav active-class hyperscript matched only the link href prefix, so
navigating to /p/<pid>/endpoints (rendered by the API Catalog handler) lost
the highlight after htmx pushState. Add a data-match attribute carrying
extra path prefixes; the API Catalog item now declares /endpoints as a
secondary match.
* Auto-format code with fourmolu
* untrack WIP incident report and migration plan drafts
These were accidentally committed in 3e6f8a2; they are local work-in-progress
notes, not source. Keep working copies on disk.
* endpoints: rewrite SQL builders with hasql-interpolate ^{} splices
Replace string-concat + rawSql glue with single multiline [HI.sql|...|]
quasiquotes using ^{sql} splices for fragments and #{val} for values.
Drop NeatInterpolation usage. Collapse countEndpointInbox default to
correct enp alias.
* DeriveUtils: use maybeToRight in refineText (hlint warning)
* hasql: derive ENUM/CITEXT decoders, widen INT/NUMERIC columns
WrappedEnumSC now takes a leading `Maybe Symbol` pinning the backing PG
type. `'Nothing` keeps the old text-backed behaviour; `('Just "schema.name")`
swaps in `D.enum`/`E.enum` so hasql-interpolate's strict OID lookup resolves
real `CREATE TYPE … AS ENUM` columns through generic DecodeRow without
per-query `::text` casts. `HI.DecodeValue (CI Text)` switches to `D.citext`
to cover the citext extension type.
Updated qualified sites: Permissions, IssueType, AnomalyTypes,
AnomalyActions, QueryLibType, KeyKind. Everything else gets `'Nothing`.
Migrations:
- 0097 drops the `email` domain (now plain citext + format CHECK) and
widens `apis.log_patterns.baseline_samples` /
`projects.replay_sessions.event_file_count` to BIGINT.
- 0098 batches the rest of the int4 → int8 widening (issues affected_*,
llm_enhancement_version, seq_num; error_patterns occurrences/etc.;
query_monitors check_interval_mins/threshold/notification_count/…;
notification_rate_limit.count) plus `apis.issues.error_rate` → float8.
Code fixes surfaced by the strict decoders:
- `Issue` gains `cooldownUntil` / `lastNotifiedAt` (rows existed since
0075/0085) and the IssueL list SQL selects them.
- BackgroundJobs SafetyNet uses `Telemetry.otelSpanColsSql` (was missing
`errors` + `message_size_bytes`).
- consumeNotificationToken returns `count` directly (no int4 cast).
- make_interval call casts hours to `int` (only signature PG provides).
- Anomalies VM format_examples now `TEXT[]` (matches Haskell Vector Text).
- LogQueries bis/cnts and totalSessions widened to Int64.
- Parser alert select emits `count(*)::float8` for Double consumers.
- Projects.downgradeToFree / upgradeToPaid cast bigint params to text for
the TEXT `order_id` comparison.
- isInCooldown SELECTs `1::bigint` so the Int64 decoder matches.
* Auto-format code with fourmolu
* 0098: combine per-table ALTERs to one rewrite each
PG rewrites the heap+TOAST+indexes once per ALTER TABLE statement
regardless of how many ALTER COLUMN clauses it contains, so collapsing
23 statements into 4 (one per table) cuts the migration's I/O by ~6x.
Notable on apis.error_patterns (8.9 GB TOAST) which goes from 9
rewrites to 1; reduces lock-hold window during prod deploy.
Also drop the Int64->Int round-trip in fetchLogPatterns: hasql's
DecodeValue Int instance already reads int8, so decoding directly into
Int (and dropping the fromIntegral) matches buildHourlyBuckets's
signature without the cast.
* tests: drop unused imports across integration specs
* fix(llm): parse criticality response directly instead of via getNormalTupleReponse
getNormalTupleReponse returns only the first line as text and the
second line as Maybe — so 'lines response' always yielded one element
and every classifyIssueCriticality call short-circuited to 'Invalid
response format from LLM'. Parse the raw LLM response with a prefix
pattern and strip each line.
* refactor: inline directionClauseSql and uncurry severityBadge
- archiveHosts/unarchiveHosts: drop let-bound dir, inline as ^{} splice
- normalLogElements: replace lambda-on-tuple with uncurry
* issues: drop unused HI.EncodeRow/ToRow from Issue and Report
Both types are written via named-column INSERTs (insertIssue, the
reports insert); the positional encoders were never exercised. Dropping
them removes a latent foot-gun now that Issue carries cooldown_until
and last_notified_at columns whose positions don't match the named
insertIssue column list.
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>1 parent 7a4b94c commit 030b103
68 files changed
Lines changed: 918 additions & 4652 deletions
File tree
- src
- Data/Effectful
- Models
- Apis
- Projects
- Telemetry
- Pages
- Charts
- Pkg
- SchemaLearning
- Web
- static/migrations
- test/integration
- BackgroundJobs
- CLI
- Opentelemetry
- Pages
- Bots
- Endpoints
- LogExplorer
- Projects
- Web
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
11 | | - | |
12 | | - | |
13 | | - | |
14 | | - | |
| 10 | + | |
15 | 11 | | |
16 | 12 | | |
17 | 13 | | |
| |||
120 | 116 | | |
121 | 117 | | |
122 | 118 | | |
123 | | - | |
| 119 | + | |
124 | 120 | | |
125 | 121 | | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
126 | 129 | | |
127 | 130 | | |
128 | 131 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
211 | 211 | | |
212 | 212 | | |
213 | 213 | | |
214 | | - | |
215 | | - | |
216 | | - | |
217 | | - | |
218 | | - | |
219 | | - | |
220 | | - | |
221 | 214 | | |
222 | 215 | | |
223 | 216 | | |
| |||
353 | 346 | | |
354 | 347 | | |
355 | 348 | | |
356 | | - | |
357 | 349 | | |
358 | 350 | | |
359 | 351 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
121 | 121 | | |
122 | 122 | | |
123 | 123 | | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
124 | 127 | | |
125 | 128 | | |
126 | 129 | | |
| |||
577 | 580 | | |
578 | 581 | | |
579 | 582 | | |
580 | | - | |
| 583 | + | |
581 | 584 | | |
582 | 585 | | |
583 | 586 | | |
| |||
663 | 666 | | |
664 | 667 | | |
665 | 668 | | |
666 | | - | |
| 669 | + | |
667 | 670 | | |
668 | 671 | | |
669 | 672 | | |
| |||
761 | 764 | | |
762 | 765 | | |
763 | 766 | | |
764 | | - | |
| 767 | + | |
765 | 768 | | |
766 | 769 | | |
767 | 770 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
362 | 362 | | |
363 | 363 | | |
364 | 364 | | |
| 365 | + | |
365 | 366 | | |
366 | 367 | | |
367 | | - | |
| 368 | + | |
368 | 369 | | |
369 | 370 | | |
370 | 371 | | |
| |||
376 | 377 | | |
377 | 378 | | |
378 | 379 | | |
| 380 | + | |
379 | 381 | | |
380 | 382 | | |
381 | | - | |
| 383 | + | |
382 | 384 | | |
383 | 385 | | |
384 | 386 | | |
| |||
441 | 443 | | |
442 | 444 | | |
443 | 445 | | |
| 446 | + | |
444 | 447 | | |
445 | 448 | | |
446 | | - | |
| 449 | + | |
447 | 450 | | |
448 | 451 | | |
449 | 452 | | |
| |||
0 commit comments