Skip to content

Develop - #211

Merged
alex-struk merged 7 commits into
mainfrom
develop
Jun 26, 2026
Merged

Develop#211
alex-struk merged 7 commits into
mainfrom
develop

Conversation

@alex-struk

Copy link
Copy Markdown
Collaborator

Summary

AI-###

Changes

Testing

Checklist

By submitting this pull request, I acknowledge that I have attempted to meet the following:

  • a self-review of my code
  • commented code particularly in hard-to-understand areas
  • corresponding changes to the documentation where required
  • changes tested to the best of my ability
  • no new errors or non-functional code

alex-struk and others added 7 commits June 26, 2026 09:36
The documents list "Failed" filter exact-matched status="failed", so
conversion_failed documents (a distinct failure state) were never shown.
Expand the filter to match both failed and conversion_failed; all other
status filters still match exactly.

Also:
- Frontend: add the missing conversion_failed badge style and include
  conversion_failed in the "Failed" stats card.
- Frontend: remove the dead rejected_by_human status (type, filter option,
  badge style, stats type) — it does not exist in the backend enum.
- Add indexes backing the list endpoint at scale: a composite
  (group_id, created_at DESC) btree for the default filter+sort, and
  pg_trgm GIN indexes on title/original_filename for the ILIKE search.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Read/layout OCR models save their output as `content` (markdown or text)
with no keyValuePairs, but the document viewer only rendered keyValuePairs,
so that output was invisible.

- Add `OcrContent` (`content`) to the frontend OcrResult type.
- New ExtractedTextView: renders OCR text with a Rendered/Raw toggle.
  Rendered markdown uses react-markdown + remark-gfm, with rehype-raw +
  rehype-sanitize so Azure's inline HTML (figures, HTML tables) renders
  safely instead of showing as literal tags.
- The viewer's OCR Results tab now shows whichever outputs exist; when
  both are present, extracted text leads and key-value fields follow.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ephemeral cleanup removes a document's blobs but retains its OCR result.
Opening such a document rendered blank, and /view returned an unhandled
500 on the dangling blob path.

- /view and /download return 410 Gone when purged_at is set, checked
  before touching blob storage; /view also wraps the blob read so an
  unexpectedly-missing blob is a 404, not a 500.
- Expose purged_at on DocumentDataDto and the frontend Document type.
- The viewer detects purged documents, skips the (failing) blob fetch,
  opens on the retained OCR data, and shows an "Original document
  removed" notice instead of a blank pane.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1. Remove the backend-services-storage (blobs) PVC and its /app/data/blobs
   mounts from backend-services and temporal-worker. The only blob providers
   are azure and minio (both remote object stores); no code path writes to the
   local volume and the live prod PVC is empty. Reclaims 2Gi in every env.

2. Pin the prod temporal Postgres data volume to 5Gi via the prod-resources
   component. Prod's pgdata PVC was grown in-place to 5Gi while the base
   manifest still requested 1.5Gi, causing the Crunchy operator to emit a
   recurring "field can not be less than previous value" reconcile error.
   base stays 1.5Gi for ephemeral/test instances.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…period

Addresses two prod memory alerts:

1. backend-services OOMKilled at the 1Gi prod limit under heavy document load.
   The earlier 512Mi->1Gi bump greatly reduced OOMs (one in 20h vs every
   ~22min); raise the prod limit to 2Gi via the prod-resources component to
   absorb the occasional spike. Limits are not quota-counted, so this is free.
   base/test stay at 512Mi.

2. Loki HighPodMemoryUsage (>90% of its 2Gi limit, set in values-openshift.yaml).
   Reduce the ingester chunk_idle_period from 30m to 5m so idle log streams
   flush to disk ~6x sooner, capping the in-memory chunk footprint instead of
   raising the limit. Applies to all instances.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Regenerated from the live backend: 132 endpoints / 183 operations.
Picks up benchmarking, client-errors, group member roles, document
stats/thumbnails and other endpoints added since last generation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…sion-failed

fix(document): purged-doc handling + OCR content + prod infra hardening
@alex-struk
alex-struk merged commit 509f70f into main Jun 26, 2026
39 of 41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant