Skip to content

fix(memory): redact inline image bytes from extraction - #4456

Open
jianxunzhang1101 wants to merge 1 commit into
volcengine:mainfrom
jianxunzhang1101:fix/redact-inline-image-tool-output
Open

fix(memory): redact inline image bytes from extraction#4456
jianxunzhang1101 wants to merge 1 commit into
volcengine:mainfrom
jianxunzhang1101:fix/redact-inline-image-tool-output

Conversation

@jianxunzhang1101

Copy link
Copy Markdown

Description

Prevent inline image Base64 from being restored into Phase 2 memory-extraction prompts as plain text. The stored session and externalized tool result remain unchanged; only the deep-copied messages used by Working Memory and long-term memory extraction receive bounded placeholders.

Human Involvement

  • A human participated in the implementation or review loop
  • This PR was generated entirely by AI agents without human participation in the loop

The reporter provided the production incident, provider errors, request-size measurements, and requested the scoped fix.

Related Issue

Fixes #4455

Related to the broader unbounded extraction-input issue #3226.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that causes existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • Performance improvement
  • Test update

Changes Made

  • Replace data:image/...;base64,... and JSON b64_json values with placeholders containing MIME type and encoded character count.
  • Apply redaction to the memory-only copy after externalized tool-output hydration, covering Working Memory and long-term extraction while preserving tool-result storage fidelity.
  • Keep a final guard in Working Memory formatting and add tests for hydrated outputs, surrounding text, b64_json, and source-message immutability.

Testing

  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have tested this on the following platforms:
    • Linux
    • macOS
    • Windows
python -m pytest -q -o addopts= tests/unit/session/test_wm_v2_guards.py
83 passed, 4 pre-existing Pydantic deprecation warnings

ruff format --check openviking/session/session.py tests/unit/session/test_wm_v2_guards.py
2 files already formatted

ruff check openviking/session/session.py tests/unit/session/test_wm_v2_guards.py
All checks passed!

git diff --check
clean

A synthetic incident-scale check used 12 tool outputs with 999,992 Base64 characters each. The raw tool text was 12,000,168 characters; the formatted Working Memory input was 1,343 characters and contained no original Base64 payload.

Checklist

  • My code follows the project's coding style
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation, or no public documentation change is required
  • My changes generate no new warnings
  • Any dependent changes have been merged and published, or none are required

Screenshots (if applicable)

Not applicable.

Additional Notes

This intentionally does not solve every oversized non-image extraction input under #3226. It removes the demonstrated inline-image failure mode without truncating or deleting the authoritative archived tool result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

[Bug]: Memory extraction rehydrates inline Base64 images as text

1 participant