Commit 50ae246
iOS composer: image attachments, attach button, autocorrect, padding (#6102)
* iOS composer: pending image attachments in store + send orchestration
Add per-terminal pending-attachment state to the mobile shell store so picked
images can be staged as drafts (keyed by terminal id, like the text draft) and
sent on the next composer submit, reusing the existing terminal.paste_image
transport.
- New MobilePendingAttachment value type (data + lowercase format + stable id),
host-testable (no UIKit).
- Store add/remove/clear/read methods plus composerCanSend (text non-empty OR
attachments present, so an images-only send is allowed).
- submitComposer() sends staged images in pick order (awaited) then the text,
then clears the staged set for the submitted terminal.
- Unit tests for add/remove/clear, per-terminal keying, send gating, and
clear-after-send.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* iOS composer UI: attach button, attachment chips, autocorrect, padding
- Replace the chevron.down (hide-composer) button to the left of the field with
a paperclip attach button that opens a PhotosUI photo picker (multi-select).
Composer dismissal still lives on the accessory toolbar's compose toggle.
- Picked images are encoded the same way the clipboard paste path encodes them
(PNG, JPEG fallback over ~8MB) and staged as pending attachments.
- Render staged attachments as a horizontal row of removable thumbnail chips
above the text field (iMessage style).
- Send is enabled when text is non-empty OR attachments are staged; send routes
through store.submitComposer() (images first, then text) and re-measures the
band height.
- Composer now uses normal text assistance (autocorrect on, sentence-case)
since it is natural language to an agent; the raw terminal input is unchanged.
- Reduce the top padding above the field (was 8pt vertical, now 2pt top / 8pt
bottom) so the composer sits tighter; band measurement still driven by
content + padding.
- Add NSPhotoLibraryUsageDescription to Info.plist and en/ja strings for the
attach and remove-attachment labels.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Refresh Swift file-length budget for this change
The touched file is already well over any reasonable size; splitting it is a
separate refactor and stored properties cannot move to an extension. Accept the
incremental growth as known debt so the budget guard reflects reality.
* Fix composer attachment send routing, failure handling, and memory
Three P1 autoreview fixes for the iOS composer image attachments:
1. Capture target terminal once in submitComposer. The image and text
sends now thread an explicit workspace + terminal id captured at submit
time, so a terminal switch while an awaited image send is in flight can
no longer reroute later images or the text to whatever is selected at
that moment. Adds internal targeted variants:
submitTerminalPasteImage(_:format:workspaceID:terminalID:) and
submitComposerInput(workspaceID:terminalID:); the public selection-based
overloads call them, preserving the clipboard-paste path.
2. Keep attachments on a failed send. The image send path now returns Bool
(threaded through sendRemoteTerminalPasteImage). submitComposer removes
each attachment only after its send is acknowledged; on a failure it
stops, keeps the remaining and failed attachments staged, and does not
submit the text, matching the text-keep-on-failure semantics.
3. Bound staging and stop per-render decodes. The picker is capped at 10
and a 32 MB total byte budget is enforced when staging. Each attachment
gets a small downsampled thumbnail built once off the main thread (via
ImageIO) and cached by id; the chip renders the cached thumbnail instead
of decoding the full Data in the view body on every keystroke.
Tests: ComposerPendingAttachmentTests updated for keep-on-failure; new
ComposerSubmitRoutingTests drives submitComposer end to end over the real
paste RPC frames (captured-terminal routing, mid-send switch, full and
partial image-send failures). Budget TSV refreshed for the two grown files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix round-2 composer-attachment autoreview findings
1. Swift 6 Sendable: PreparedAttachment no longer carries a non-Sendable
UIImage across the detached thumbnail-prep task. The off-main path keeps
ImageIO downsampling but returns the thumbnail as PNG Data; the UIImage is
built on the main actor when populating the chip cache.
2. Privacy: clear pendingAttachmentsByTerminalID in signOut(), alongside the
text-draft wipe, so a previous account's staged photo bytes cannot resurface
on a reused terminal id.
3. Re-entrancy: add isSubmittingComposer guard around the whole submitComposer()
(images + text), so a double tap on Send cannot re-upload the still-staged
attachments while the first RPC awaits. Failure still keeps attachments.
4. UI test: testComposerSurvivesRepeatedOpenCloseCycles closes the composer via
the accessory compose toggle instead of the removed MobileComposerClose
chevron; drop the now-unused bandClose constant.
5. Localize NSPhotoLibraryUsageDescription in InfoPlist.xcstrings (en + ja).
Add host tests for sign-out clearing (finding 2) and double-submit guard
(finding 3). Refresh the Swift file-length budget for the two touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Fix round-3 composer-attachment autoreview findings
Two async/privacy bugs from the codex autoreview:
1. submitComposer captured the target terminal but re-read the live
terminalInputText after the awaited image paste_image RPCs. A terminal
switch (draft swap) or a field edit during those awaits could skip the
composed text or paste a different terminal's draft. Snapshot the text at
the very start of submitComposer (before any await) and thread it through
submitComposerInput via a new capturedText param; the post-send reconcile
already keys on that same sent text, so a mid-send edit is preserved
(cleared only when the field still equals the snapshot). Text-only entry
points keep nil (no prior await, no drift). Images-only sends snapshot
empty text and no-op the text submit.
2. The photo picker started an unstructured Task that awaited load+encode then
unconditionally re-staged bytes via addPendingAttachment. A sign-out in
flight cleared pending attachments but the continuation re-added the
previous user's photo under an explicit terminal id. Add a signInGeneration
token bumped by signOut, captured before staging, and re-checked in a new
guarded addPendingAttachment(...ifSessionGeneration:) store path that drops
the result when the token moved or the target terminal no longer exists.
Tests: ComposerSubmitRoutingTests gains text-snapshot-survives-edit and
text-snapshot-survives-switch; ComposerPendingAttachmentTests gains
guarded-add dropped-after-signout, dropped-when-terminal-gone, and
succeeds-when-unchanged. Budget actuals refreshed for both touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Harden composer image-attach staging (round 3)
Fix three autoreview findings in the iOS composer image-attachment
staging and send path.
Finding 1 (OOM): replace the full-raster pngData() encode with a bounded
ImageIO path. prepare() now downsamples the picked item via
CGImageSourceCreateThumbnailAtIndex (send payload capped at 2048 px
longest edge, thumbnail at 168 px) and re-encodes the bounded CGImage
through CGImageDestination, so a large HEIC/JPEG/panorama is never
materialized as a full-resolution raster. The send payload tries PNG at
the send size, then JPEG at decreasing quality, then smaller dimensions,
all enforced under the 8 MB per-image cap. Returns Data only across the
concurrency boundary (Sendable-safe).
Finding 2 (caps bypass): enforce the count cap (10), total-byte budget
(32 MB), and per-image cap (8 MB) atomically inside the store's
addPendingAttachment, computed against the current staged set at mutation
time on the MainActor. Two racing picker batches can no longer both append
past the cap. The view now tracks the staging Task and cancels the prior
one when a new batch starts, and treats the store as the authoritative cap
(its own checks are only a pre-filter). Sign-in generation guard kept.
Finding 3 (removed-but-uploaded): submitComposer re-checks each attachment
is still staged for the captured terminal before uploading it, so a chip
the user deletes mid-send is skipped instead of uploaded from the local
snapshot. Failure-keeps-attachments and re-entrancy behavior unchanged.
Tests: store-enforced count and byte caps (including racing adds against
the same starting budget) and a mid-send removal skipping that attachment.
Budget TSV bumped for the two touched files.
* Harden composer image-attach memory: topology prune + file-backed picker load
Finding 1: the store could retain orphaned photo Data for stale terminal ids.
The base addPendingAttachment now validates the target id exists in the current
topology (shared terminalExistsInTopology check, used by both add paths so no
unchecked entry point remains), and the workspaces didSet prunes
pendingAttachmentsByTerminalID for any terminal that disappears from topology so
multi-MB staged bytes are released on a sync rather than held until sign-out.
Sign-out clear is unchanged. Added tests: base-path add for a missing id is
rejected; a topology update dropping a terminal prunes its attachments while a
surviving (and a moved-elsewhere) terminal keeps its.
Finding 2: the picker loaded the full original asset into memory before any cap.
Replaced loadTransferable(Data) with a file-backed ImportedImageFile Transferable
(FileRepresentation(contentType: .image)) that copies the import to a temp URL,
so a huge ProRAW/DNG/panorama never enters memory as Data. The picker now
size-gates the file on disk (60 MB raw bound, above the 8 MB per-image cap since
HEIC decodes larger) before reading, downsamples straight from the URL via
CGImageSourceCreateWithURL, and deletes the temp file after encoding. Per-image
and total caps, atomic store enforcement, the session-generation guard, and the
staging Task cancellation are all preserved. Only Data/String/URL cross
concurrency boundaries.
Refreshed the swift file-length budget for the two touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Bound pending image bytes globally; make picker ImageIO cancellable
Finding 1: pending attachment bytes were capped per terminal but unbounded
across all terminals, so staging photos across many terminals/workspaces grew
linearly with terminal count and could OOM. Add global all-terminals caps
(maxPendingAttachmentTotalBytesAllTerminals = 64 MB,
maxPendingAttachmentCountAllTerminals = 20) enforced atomically in
addPendingAttachment after the per-terminal checks, as a hard reject. The sum
across all keys is consistent because the add runs on @mainactor. Per-terminal
caps stay. Host tests cover the multi-terminal case (global byte and count
budgets bind while each terminal is under its own cap; per-terminal cap still
binds under global headroom).
Finding 2: the picker's ImageIO decode/encode ran in an unstructured
Task.detached that did not inherit cancellation, so cancelling the staging task
(re-pick, terminal switch, view disappear) left the decode running and fanning
out temp files. Run prepare() in a structured background-priority child task
group so cancellation propagates; check Task.isCancelled before launching and
before the heavy CGImageSourceCreateWithURL and the thumbnail encode. Cancel the
staging task on composer .onDisappear and on a terminalID change, in addition to
the existing cancel-on-new-batch. Temp files are still cleaned on the
cancellation path via the existing per-iteration defer. Only Sendable Data
crosses concurrency boundaries.
Bump swift-file-length-budget.tsv for the two touched files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* iOS composer: abort submit on session/connection change mid-send
submitComposer() snapshots attachments and text up front but awaits a
paste_image RPC per image. sendRemoteTerminalPasteImage returns true even
when a superseded connection answered, so a sign-out, account switch, Mac
switch, or reconnect that landed during an image await let the loop keep
going: the next staged image and then the captured text were sent through
whatever remoteClient is now current, leaking the previous user's or
previous Mac's unsent content into a different session.
Capture signInGeneration (sign-out / account switch) and connectionGeneration
(Mac switch / reconnect / disconnect) at the start of the run and re-check
both before every image send and before the text send via
isComposerSubmitIdentityCurrent. On mismatch, abort the whole submit (stop
the loop, do not send the text) and leave attachments and text staged for a
retry. Both generations already existed and are bumped on those edges.
Add host-testable coverage in ComposerSubmitRoutingTests: a sign-out and a
connection swap between the first and second image send each abort the
submit so no further image and no text reach the new session, and the
connection-swap case keeps the unsent attachment and text staged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: cmux-lawrence <cmux-lawrence@cmux-lawrences-Mac-mini.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>1 parent 75525e6 commit 50ae246
11 files changed
Lines changed: 2276 additions & 97 deletions
File tree
- .github
- Packages
- CmuxMobileShellModel/Sources/CmuxMobileShellModel
- CmuxMobileShellUI/Sources/CmuxMobileShellUI
- CmuxMobileShell
- Sources/CmuxMobileShell
- Tests/CmuxMobileShellTests
- ios
- Config
- cmuxUITests
- cmux/Resources
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| |||
212 | 212 | | |
213 | 213 | | |
214 | 214 | | |
| 215 | + | |
Lines changed: 462 additions & 24 deletions
Large diffs are not rendered by default.
Lines changed: 463 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 350 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
| 266 | + | |
| 267 | + | |
| 268 | + | |
| 269 | + | |
| 270 | + | |
| 271 | + | |
| 272 | + | |
| 273 | + | |
| 274 | + | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
| 278 | + | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
0 commit comments