Skip to content

Attempt forced restoration for locally missing attachments - #837

Draft
qayshp wants to merge 3 commits into
BlueBubblesApp:developmentfrom
qayshp:agent/force-download-missing-attachment
Draft

Attempt forced restoration for locally missing attachments#837
qayshp wants to merge 3 commits into
BlueBubblesApp:developmentfrom
qayshp:agent/force-download-missing-attachment

Conversation

@qayshp

@qayshp qayshp commented Jul 23, 2026

Copy link
Copy Markdown

Summary

  • return 404 Not Found immediately when no attachment database row exists
  • when a row exists but its local file is missing, attempt forceDownload only when forcing is allowed and the Private API is enabled
  • continue with the refreshed attachment row so a restored path and MIME type can be streamed
  • preserve a clean server error when restoration fails or the file remains absent
  • add focused mocked tests for success, absent rows, force=false, disabled Private API, incomplete restoration, and a rejected forceDownload

Why

The regular attachment download endpoint defaults force to true, but its recovery branch was attached to the wrong condition. It ran only when the database lookup returned no attachment, even though AttachmentInterface.forceDownload requires an existing attachment object. The common case—an intact database row whose local file was purged—therefore never reached the restoration path.

Behavior

For an eligible missing local file, the endpoint asks the Private API helper to restore the attachment and uses the refreshed row if a file becomes available. Unknown GUIDs remain a deterministic 404; force=false and disabled-Private-API requests do not attempt restoration. A rejected or incomplete restoration logs safely, creates no file stream, and returns the existing clean missing-disk 500.

Validation

Validated on current head 58f57f95:

  • npm run test:attachment-download --workspace @bluebubbles/server — 6/6 tests passed
  • the committed rejection-path test verifies one sanitized GUID-bearing log, no FileStream, and the clean 500 fallback
  • targeted ESLint, Prettier, and git diff --check passed

Live-validation limitation

This is not an end-to-end live-restoration pass. On a SIP-disabled Intel Sequoia host, a naturally purged incoming attachment reached the forced-restoration path and Private API helper, but did not restore within the 10-minute completion wait. The request returned the expected clean fallback 500; the database row remained incomplete and the local file remained absent. This validates routing and fallback behavior, not successful restoration in that environment.

The broader server type-check still reaches the pre-existing unrelated ScheduledService.ts:39 NodeJS.Timer/clearInterval error.

Fixes #749

@qayshp

qayshp commented Jul 23, 2026

Copy link
Copy Markdown
Author

Validation update

Validated the safe paths on SIP-enabled arm64 macOS 26.5.2 with Node 20.11.1 and npm 10.2.4.

  • npm run test:attachment-download --workspace=@bluebubbles/server — 5/5 committed tests passed.
  • A separate no-file-change test harness exercised a rejected forceDownload: it returned the same clean disk-missing 500 response, emitted one GUID-bearing log entry, and created no FileStream.

That rejection case was not added to the committed test suite, so the review follow-up remains open. A real purged-attachment restore is also deferred to a SIP-disabled Mac with the Private API enabled and connected; that validation will perform an actual iCloud attachment download.

No messages were sent or received during this validation. No screenshot is needed because the relevant evidence is the HTTP response, stream behavior, and sanitized server log.

@qayshp

qayshp commented Jul 23, 2026

Copy link
Copy Markdown
Author

Remote validation follow-up for f9437c49

Validated on an Intel Sequoia host with a custom SIP configuration sufficient for the existing Private API helper. The server reported Private API enabled, the helper connected, and an iMessage account available.

  • Ran the PR-focused attachment-download suite on the remote host: 5/5 tests passed.
  • Also verified that a runtime built from this commit could reach API-ready state. Two local, uncommitted build-only workarounds were needed for pre-existing toolchain issues; neither changed attachment restoration code.
  • After announcing the real action, created the designated two-recipient conversation and sent one starter message so a fixture could be attempted.
  • The legacy attachment-send action timed out. A multipart attempt returned HTTP 200, but database checks showed 0 linked attachment records, 0 completed/state-5 fixture rows, and the generated upload rows remained unlinked in state 0.

Because no genuine linked, remotely restorable/purged attachment was produced, I did not run or claim the live force=falseforce=trueforce=false restoration sequence. Doing so against the unlinked rows would be a false positive.

The exact staged fixture files were moved to Trash, temporary build/runtime material was removed, and the installed server was restored and rechecked healthy with its helper connected.

To complete the live restoration test, the designated conversation needs a genuine attachment that Messages has successfully linked and made remotely restorable—for example, one participant can send a small attachment first, after which the purge/restore sequence can be exercised.

@qayshp

qayshp commented Jul 24, 2026

Copy link
Copy Markdown
Author

Live SIP-disabled validation follow-up for f9437c49

Tested on an Intel Sequoia host with a custom SIP configuration sufficient for the existing Private API helper, using one naturally purged incoming attachment from an authorized existing iMessage conversation.

  • The pinned database row remained uniquely selectable and eligible before the attempt.
  • force=false&original=true returned the expected missing-disk 500 baseline.
  • force=true&original=true reached the forced-restoration path and Private API helper, then exhausted the 10-minute completion wait.
  • The request returned 500 with the static fallback category Attachment does not exist on disk.
  • Afterward, the same pinned row remained in transfer state 0; it did not reach state 5, and the local file remained absent.

This is not a live-restoration pass. The result is attributable to helper/environment/fixture behavior: the attachment was not restored within the allowed wait. It does not demonstrate a defect in this PR's routing change, and the final 500 is the PR's expected fallback when restoration does not complete.

A separate private test-orchestration guard also rejected macOS's resolved Python executable path instead of the literal /usr/bin/python3. The harness and request were allowed to finish naturally before cleanup; that guard issue is unrelated to the PR code. The one-shot restore marker was claimed, so the operation was not retried.

No message was sent or received. No conversation, participant, attachment identifier, filename, path, type, size, date, content, hash, or screenshot was posted or retained in this validation note. The installed 1.9.9 server was restored and passed two consecutive checks for exact identity, authenticated health, Private API/helper readiness, account readiness, and sole listener ownership.

@qayshp qayshp changed the title Restore purged attachments during download Attempt forced restoration for locally missing attachments Jul 27, 2026
@qayshp

qayshp commented Jul 27, 2026

Copy link
Copy Markdown
Author

Superseding the earlier validation notes: current head 58f57f95 contains the committed rejected-forceDownload coverage, and the focused suite re-passed 6/6. The rejection case verifies one sanitized log entry, no file stream, and the clean missing-disk 500.

Live restoration is still not a pass: a naturally purged attachment reached the forced-restoration path and Private API helper on the SIP-disabled test host, but did not restore within 10 minutes. It remained locally absent and the request returned the expected fallback 500.

@qayshp
qayshp marked this pull request as ready for review July 28, 2026 07:02
@qayshp
qayshp marked this pull request as draft July 28, 2026 07:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant