Skip to content

feat(core): activate WebP screenshots with JPEG fallback - #3014

Draft
quanru wants to merge 9 commits into
feat/webp-format-aware-consumers-v2from
feat/webp-core-activation-v2
Draft

feat(core): activate WebP screenshots with JPEG fallback#3014
quanru wants to merge 9 commits into
feat/webp-format-aware-consumers-v2from
feat/webp-core-activation-v2

Conversation

@quanru

@quanru quanru commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • centralize Core screenshot preparation and encode pixel-changing screenshot paths as WebP
  • keep screenshot producers on their native capture formats
  • send inline model images as WebP by default, with an explicit JPEG compatibility fallback
  • deduplicate identical image conversions within one model request using a request-scoped cache
  • document model image input configuration in English and Chinese

Behavior

  • MIDSCENE_MODEL_IMAGE_INPUT_FORMAT=webp is the default
  • MIDSCENE_MODEL_IMAGE_INPUT_FORMAT=jpeg enables compatibility for providers without WebP support
  • existing images already in the target format are reused without re-encoding
  • duplicate inline images are converted once per model request
  • remote image URLs remain unchanged

Validation

  • pnpm run lint
  • pnpm --dir packages/shared test (524 passed)
  • pnpm --dir packages/core test (1518 passed, 8 skipped)
  • pnpm exec nx build @midscene/shared
  • pnpm exec nx build @midscene/core
  • pnpm exec nx build @midscene/report
  • pnpm --dir packages/web-integration test -- tests/unit-test/browser-webp-encoder-e2e.test.ts (2 passed)
  • pnpm --dir packages/core exec vitest --run tests/unit-test/model-image-input.test.ts (4 passed)

Stack

The superseded WebP stacks (#2851#2855 and #2857#2859) are closed. #2856 remains an independent model-input evidence change.

quanru and others added 2 commits August 20, 2026 22:44
Co-authored-by: quanruzhuoxiu <robin.lifeos@gmail.com>
Co-authored-by: quanruzhuoxiu <robin.lifeos@gmail.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 20, 2026

Copy link
Copy Markdown

Deploying midscene with  Cloudflare Pages  Cloudflare Pages

Latest commit: 23a48d2
Status: ✅  Deploy successful!
Preview URL: https://2ab52bf6.midscene.pages.dev
Branch Preview URL: https://feat-webp-core-activation-v2.midscene.pages.dev

View logs

quanru and others added 2 commits August 24, 2026 15:14
Co-authored-by: quanruzhuoxiu <robin.lifeos@gmail.com>
Co-authored-by: quanruzhuoxiu <robin.lifeos@gmail.com>
quanru and others added 3 commits August 26, 2026 17:50
Merge the validated format-aware screenshot layer into Core activation.

Co-authored-by: quanruzhuoxiu <robin.lifeos@gmail.com>
Cache identical inline image conversions for one model request.

Keep the cache request-scoped to bound memory and preserve retry behavior.

Co-authored-by: quanruzhuoxiu <robin.lifeos@gmail.com>
Merge the Rstest browser encoder test fix through the final stack layer.

Co-authored-by: quanruzhuoxiu <robin.lifeos@gmail.com>
@quanru
quanru marked this pull request as draft August 27, 2026 09:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant