Skip to content

Ai 1264 - #179

Merged
kmandryk merged 17 commits into
developfrom
AI-1264
Jun 17, 2026
Merged

Ai 1264#179
kmandryk merged 17 commits into
developfrom
AI-1264

Conversation

@kmandryk

@kmandryk kmandryk commented Jun 4, 2026

Copy link
Copy Markdown
Collaborator

Summary

Optimize temporal db storage to avoid bloat

AI-1264

Changes

Testing

Checklist

By submitting this pull request, I acknowledge that I have attempted to meet the following:

  • a self-review of my code
  • commented code particularly in hard-to-understand areas
  • corresponding changes to the documentation where required
  • changes tested to the best of my ability
  • no new errors or non-functional code

kmandryk added 10 commits May 25, 2026 16:42
Introduce @ai-di/temporal-payload-codec so workers and clients compress
workflow payloads consistently while preserving inner encoding metadata.
Add @ai-di/graph-workflow-config for computeConfigHash and
applyWorkflowConfigOverrides, wired from temporal and backend-services.
Store large OCR JSON in blob storage and pass OcrPayloadRef through
activities; add jest setup for activity logger mocks.
Read and write OCR payloads via OcrPayloadRef in poll, extract, enrich,
cleanup, confidence, mistral, and post-OCR correction activities.
graphWorkflow resolves config via activity by versionId and configHash;
wire gzip payload codec on worker; slim child workflow args and results.
Benchmark sample workflows pass workflowVersionId, configHash, and
groupId; materialize dataset returns groupId for synthetic documents.
Temporal client starts graphWorkflow with workflowVersionId, merged
configHash, and overrides; benchmark and ground-truth paths aligned.
CLI migrates workflow configs to ocrResponseRef/ocrResultRef keys;
seed and template JSON updated for ref-based ctx ports.
Align workflow integration test scripts and starter with versioned
graphWorkflow input (workflowVersionId and configHash).
Comment thread packages/graph-workflow-config/src/workflow-config-overrides.ts Fixed
Reject unsafe dot-path segments to prevent prototype pollution when applying workflow config overrides. Add a UNSAFE_PATH_SEGMENTS check and assertSafePathSegments in packages/graph-workflow-config, call it from setNestedValue, and tighten property checks using Object.prototype.hasOwnProperty.call. Add unit tests in apps/backend-services to ensure __proto__ and constructor segments are rejected. Update docs to document the security behavior and note that benchmark APIs further whitelist overrides against exposedParams.
Comment thread packages/graph-workflow-config/src/workflow-config-overrides.ts
Harden applyWorkflowConfigOverrides against prototype pollution and invalid path segments. Replace JSON.parse-based cloning with deepCloneToNullPrototype to produce null-prototype objects, so nested assignments cannot reach Object.prototype. Add SAFE_PATH_SEGMENT regex and isSafeOverridePathSegment to validate dot-path segments (reject blocklisted names, empty segments, and non-identifier segments), and update assertSafePathSegments to use it. Make setNestedValue create null-prototype objects for intermediate nodes. Update tests to use Object.hasOwn and add a test rejecting non-identifier segments; update docs to reflect the stricter rules.

@dbarkowsky dbarkowsky left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No large concerns. Up to you if you want to fix that Copilot-found error and the one issue I noted.

Comment thread apps/temporal/src/activities/poll-ocr-results.ts
kmandryk and others added 4 commits June 9, 2026 13:53
…function'

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
Treat failed Azure OCR responses as non-retryable Temporal failures. Add ApplicationFailure import and a throwFailedOcrResponse helper that builds an ApplicationFailure with a descriptive message (including error code/message) and includes the OCR response in details. Replace previous 'failed' return branches with this helper and update tests to expect ApplicationFailure with nonRetryable=true and the response details.
@kmandryk
kmandryk merged commit cf89662 into develop Jun 17, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants