fix(document): purged-doc handling + OCR content + prod infra hardening - #210
Merged
alex-struk merged 6 commits intoJun 26, 2026
Merged
Conversation
The documents list "Failed" filter exact-matched status="failed", so conversion_failed documents (a distinct failure state) were never shown. Expand the filter to match both failed and conversion_failed; all other status filters still match exactly. Also: - Frontend: add the missing conversion_failed badge style and include conversion_failed in the "Failed" stats card. - Frontend: remove the dead rejected_by_human status (type, filter option, badge style, stats type) — it does not exist in the backend enum. - Add indexes backing the list endpoint at scale: a composite (group_id, created_at DESC) btree for the default filter+sort, and pg_trgm GIN indexes on title/original_filename for the ILIKE search. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Read/layout OCR models save their output as `content` (markdown or text) with no keyValuePairs, but the document viewer only rendered keyValuePairs, so that output was invisible. - Add `OcrContent` (`content`) to the frontend OcrResult type. - New ExtractedTextView: renders OCR text with a Rendered/Raw toggle. Rendered markdown uses react-markdown + remark-gfm, with rehype-raw + rehype-sanitize so Azure's inline HTML (figures, HTML tables) renders safely instead of showing as literal tags. - The viewer's OCR Results tab now shows whichever outputs exist; when both are present, extracted text leads and key-value fields follow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ephemeral cleanup removes a document's blobs but retains its OCR result. Opening such a document rendered blank, and /view returned an unhandled 500 on the dangling blob path. - /view and /download return 410 Gone when purged_at is set, checked before touching blob storage; /view also wraps the blob read so an unexpectedly-missing blob is a 404, not a 500. - Expose purged_at on DocumentDataDto and the frontend Document type. - The viewer detects purged documents, skips the (failing) blob fetch, opens on the retained OCR data, and shows an "Original document removed" notice instead of a blank pane. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1. Remove the backend-services-storage (blobs) PVC and its /app/data/blobs mounts from backend-services and temporal-worker. The only blob providers are azure and minio (both remote object stores); no code path writes to the local volume and the live prod PVC is empty. Reclaims 2Gi in every env. 2. Pin the prod temporal Postgres data volume to 5Gi via the prod-resources component. Prod's pgdata PVC was grown in-place to 5Gi while the base manifest still requested 1.5Gi, causing the Crunchy operator to emit a recurring "field can not be less than previous value" reconcile error. base stays 1.5Gi for ephemeral/test instances. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…period Addresses two prod memory alerts: 1. backend-services OOMKilled at the 1Gi prod limit under heavy document load. The earlier 512Mi->1Gi bump greatly reduced OOMs (one in 20h vs every ~22min); raise the prod limit to 2Gi via the prod-resources component to absorb the occasional spike. Limits are not quota-counted, so this is free. base/test stay at 512Mi. 2. Loki HighPodMemoryUsage (>90% of its 2Gi limit, set in values-openshift.yaml). Reduce the ingester chunk_idle_period from 30m to 5m so idle log streams flush to disk ~6x sooner, capping the in-memory chunk footprint instead of raising the limit. Applies to all instances. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Regenerated from the live backend: 132 endpoints / 183 operations. Picks up benchmarking, client-errors, group member roles, document stats/thumbnails and other endpoints added since last generation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
alex-struk
marked this pull request as ready for review
June 26, 2026 18:41
alex-struk
requested review from
NoorChasib,
antsand,
dbarkowsky and
kmandryk
as code owners
June 26, 2026 18:41
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft PR covering the document-viewer fixes on this branch plus several prod-infra hardening changes made while responding to OpenShift memory/storage alerts.
Document changes
4dba402efix(document): includeconversion_failedin "failed" list filter — the "failed" status filter now also surfaces documents inconversion_failed.c617cdbafeat(document): surface OCR markdown/text content in the viewer — exposes OCR markdown/text content through the viewer.82463ed4feat(document): handle purged documents in viewer and content APIs — graceful handling of purged documents in the viewer and content APIs.Infra: prod memory & storage hardening
Made in response to
PodInErrorState/HighPodMemoryUsagealerts infd34fb-prod.9185d909remove unused blobs PVC + fix temporal-pg storage driftbackend-services-storage(blobs) PVC and its/app/data/blobsmounts from backend-services and temporal-worker. The only blob providers areazureandminio(both remote object stores); no code path writes to the local volume and the live prod PVC is empty. Reclaims 2Gi in every env (delete the live PVC manually after deploy).prod-resourcescomponent. Prod's pgdata PVC was grown in-place to 5Gi while base still requested 1.5Gi, causing a recurring Crunchy "field can not be less than previous value" reconcile error. base stays 1.5Gi for ephemeral/test.71dd9fa9raise prod backend memory to 2Gi + cut Loki chunk idle periodprod-resourcescomponent (absorbs occasional OOM spikes under heavy document load; limits aren't quota-counted, so free; base/test stay 512Mi).chunk_idle_period30m → 5m so idle log streams flush to disk ~6× sooner, capping in-memory footprint (mitigates Loki high-memory alert) without raising the limit. Applies to all instances.Notes / follow-ups (not in this PR)
prod-resourcesKustomize component; requests are unchanged so there is no quota impact.oc delete pvc backend-services-storage(post-deploy), plus orphaneddata-feature-deployment-f-plg-*PVCs (~4Gi) and stale pgBackRest backup job pods.🤖 Generated with Claude Code