Skip to content

fix(ios): reject blank native capture frames - #55

Merged
Chinmay-KB merged 3 commits into
mainfrom
codex/fix-ios-native-blank-frames
Sep 2, 2026
Merged

fix(ios): reject blank native capture frames#55
Chinmay-KB merged 3 commits into
mainfrom
codex/fix-ios-native-blank-frames

Conversation

@Chinmay-KB

@Chinmay-KB Chinmay-KB commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • reject transparent and near-white native iOS frames
  • keep engine-surface failure explicit as pixelCopyFailed
  • validate explicit native view-hierarchy captures before encoding
  • bound blank-frame detection to 4,096 sampled pixels
  • publish the Apple runtime as 0.1.1

Physical-device evidence

  • Blend 3.17.187 (1482) on iPhone 12, iOS 18.6.2
  • 10 native view-hierarchy frames received by PMKit recorder
  • no white frames
  • recorder verdict: ok

Verification

  • 15 Apple XCTest cases passed
  • CocoaPods lint passed for TugboatCaptureRuntime 0.1.1
  • Swift public API check passed
  • all 10 GitHub checks passed

The Flutter package will explicitly select the hierarchy retry and increase degraded screenshot resolution after CocoaPods trunk publishes 0.1.1.

@Chinmay-KB
Chinmay-KB requested a lite review from Copilot September 2, 2026 14:52
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 2, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-02T14:55:25.729180Z d37b4c6 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d37b4c6449

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread platforms/apple/Sources/TugboatCaptureRuntime/Internal/AppleViewCapture.swift Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The updated behavior is not fully reflected in the new documentation text (and a related code comment is misleading), which should be corrected to keep release notes aligned with actual runtime behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates the Apple capture runtime to avoid returning “blank” native frames by detecting transparent/near-white outputs, retrying engine-surface captures with view-hierarchy capture when needed, and reporting the actual coverage that produced the returned frame.

Changes:

  • Added pixel-content validation to reject transparent/near-white captures and added an engine-surface → view-hierarchy retry path.
  • Propagated the actual capture coverage through to CaptureResult and expanded XCTest coverage for the new behaviors.
  • Bumped TugboatCaptureRuntime CocoaPods version to 0.1.1 and documented the runtime behavior in the compatibility table.
File summaries
File Description
TugboatCaptureRuntime.podspec Bumps the CocoaPods version to 0.1.1.
platforms/apple/Tests/TugboatCaptureRuntimeTests/CaptureRuntimeModeTests.swift Adds tests for retrying blank engine-surface captures and rejecting blank/white outputs.
platforms/apple/Sources/TugboatCaptureRuntime/Internal/AppleViewCapture.swift Implements blank-frame detection, retry logic, and tracks the coverage used for the returned bitmap.
platforms/apple/Sources/TugboatCaptureRuntime/CaptureRuntime.swift Returns the bitmap’s actual coverage and improves surface-availability detection before reporting failures.
docs/releases/compatibility.md Documents the 0.1.1 Apple runtime behavior in the compatibility table.
Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 3
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread docs/releases/compatibility.md Outdated
Comment thread platforms/apple/Sources/TugboatCaptureRuntime/Internal/AppleViewCapture.swift Outdated
@Chinmay-KB
Chinmay-KB requested a lite review from Copilot and removed request for Copilot September 2, 2026 15:02
@Chinmay-KB
Chinmay-KB requested a lite review from Copilot and removed request for Copilot September 2, 2026 15:03
@Chinmay-KB
Chinmay-KB merged commit 0c53cf9 into main Sep 2, 2026
12 checks passed

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The implementation behavior conflicts with the PR description around engine-surface fallback, and the new blank-frame detection should tighten its alpha/visibility heuristic to avoid accepting effectively transparent frames.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details
  • Files reviewed: 5/5 changed files
  • Comments generated: 2
  • Review effort level: Lite

let y = linearIndex / width
let x = linearIndex % width
let pixel = bytes.advanced(by: y * stride + x * 4)
let isVisible = pixel[3] != 0
Comment on lines +76 to +80
pixelWidth: pixelWidth,
pixelHeight: pixelHeight,
coverage: .engineSurface,
afterScreenUpdates: false
), bitmap.hasCapturedContent else { return nil }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants