refactor(s3): move image upload from frontend to backend

[OhYee]

Part of #145. Depends on #163, please review/merge that first.

Why

The earlier PR shipped a frontend-direct-upload flow (presigned URLs). Three operational issues led us to move uploading to the backend:

1. Single auth boundary — presigned URLs bypass app-level access checks once issued.
2. Coverage — paste / drag / drop / import flow through different code paths; the frontend addFiles() hook misses Excalidraw 0.18+'s internal paste path. Server-side processing on save catches every code path uniformly.
3. Auditability — S3File rows are written by the backend in the same handler as the drawing row; failure modes are surfaceable.

What

• New backend/src/fileProcessing.ts exposes processFilesForS3(files, userId, drawingId, prisma) — scans the file record for base64 dataURLs, uploads them, upserts a row keyed (drawingId, fileId), returns a rewritten file record.
• s3.ts adds uploadBuffer(), listS3Objects(), copyS3Object().
• Wired into POST /drawings, PUT /drawings/:id, and both /import/* routes.
• Removed: POST /files/upload-url (no longer used by any client).
• Removed: frontend S3 upload code (addFiles monkey-patch, pendingS3UploadsRef, the save-time await loop).

Notable safety fixes (review feedback, separate commits)

• fix(s3): validate fileId before using as S3 key segment — defence-in-depth on the upload path (sanitiser layer is in feat(s3): add S3 image upload support for Excalidraw drawings #163).
• fix(drawings): version pre-check, non-owner gate, DELETE-time S3 cleanup
  • PUT does a cheap version check before uploading. Otherwise on 409 stale-version we'd already have PUT every base64 file to S3 with no DB row.
  • Non-owner editors (link-share + edit, share-with-user) cannot add new fileIds — prevents a shared editor from filling the owner's bucket via 50 MB × N uploads.
  • DELETE handler purges S3 objects + S3File rows under the per-drawing prefix (without this, deleting a drawing leaks every upload forever).
• fix(import): pre-resolve drawing id, bound parallel S3 uploads — on imported-id collision the old code uploaded under prepared.id then wrote the row under uuidv4(), leaving every uploaded object as a permanent orphan. Pre-resolves the final id before upload. Also batches parallel uploads (concurrency 8) so thousands-of-drawings imports don't hit reverse-proxy timeouts.
• fix(drawings): copy S3 storage on duplicate so the original can be deleted — Duplicate route now does a server-side CopyObjectCommand for each S3File row of the original under the new drawingId path. Without this, deleting the original would silently break every image in the duplicate's scene (since the duplicate references the original's prefix).
• fix(drawings): rewrite preview SVG so it doesn't inline base64 after S3 upload — the frontend generates the preview SVG before the round-trip uploads files; the SVG embeds the base64 dataURL as the <image href>. After this fix, the backend rewrites the preview string with the same originalDataURL → s3URL substitutions it applied to Drawing.files, so previews don't carry the inlined base64 forever.

Risk

• ⚠️ API breaking: POST /files/upload-url removed. Old browser tabs holding a stale frontend will hit 404 on paste, but processFilesForS3 still accepts base64 in the save payload, so the save itself succeeds — user-visible damage is one console error per paste until refresh.
• Schema unchanged from feat(s3): add S3 image upload support for Excalidraw drawings #163.

Verification

• npm run build, npm test (172 / 172 pass)
• Tested against Alibaba Cloud OSS: paste, drop, save, duplicate, delete, share-link viewing of images all work end-to-end.

[Copilot]

Pull request overview

This PR refactors ExcaliDash’s S3 image storage flow by moving image upload responsibility from the frontend to the backend save/import handlers, aiming to unify coverage across paste/drag/drop/import paths and improve operational auditability/cleanup.

Changes:

• Added backend-side file processing (processFilesForS3) to detect base64 dataURLs, upload to S3, upsert S3File records, and rewrite Drawing.files (plus preview SVG rewriting).
• Introduced backend S3 helpers and new /files/config + /files/:drawingId/:fileId download redirect route; integrated S3 cleanup/copy behavior into drawing delete/duplicate flows.
• Updated import routes to pre-resolve drawing IDs and batch S3 uploads with bounded concurrency.

Reviewed changes

Copilot reviewed 19 out of 20 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
frontend/src/pages/Editor.tsx	Updates editor save/sync behavior; adds initial file-id tracking (currently unused) and adjusts file sync bookkeeping.
frontend/src/api/index.ts	Adds a cached /files/config probe helper (isS3Enabled).
backend/tsconfig.json	Excludes test files from backend TS compilation inputs.
backend/src/security.ts	Tightens sanitization to drop unsafe file IDs and allow S3/CDN URLs + /api/files/:drawingId/:fileId paths.
backend/src/s3.ts	Adds backend S3 client helpers (upload buffer, list, copy) and URL helpers.
backend/src/fileProcessing.ts	Implements backend-side base64 scanning, S3 upload, S3File upsert, and preview SVG rewriting.
backend/src/routes/files.ts	Adds S3-related routes: /files/config and /files/:drawingId/:fileId redirect for private-bucket mode.
backend/src/routes/dashboard/types.ts	Extends dashboard route deps to include processFilesForS3.
backend/src/routes/dashboard/drawings.ts	Integrates backend uploads into create/update, adds delete-time cleanup, and duplicate-time S3 copy + URL rewrite.
backend/src/routes/importExport/excalidashImportRoutes.ts	Pre-resolves final drawing IDs and processes files through backend upload path with bounded concurrency.
backend/src/routes/importExport/legacySqliteImportRoutes.ts	Same as above for legacy SQLite imports; writes processed files into DB.
backend/src/index.ts	Initializes S3 from env config, registers file routes, and wires processFilesForS3 into dashboard routes.
backend/src/config.ts	Adds backend config parsing for S3-related env vars.
backend/src/tests/fileProcessing.test.ts	Adds unit tests for decodeDataURL and processFilesForS3.
backend/prisma/schema.prisma	Defines S3File with composite primary key (drawingId, fileId).
backend/prisma/migrations/20260408060000_add_s3_files/migration.sql	Initial migration creating the original single-column PK S3File table.
backend/prisma/migrations/20260506130000_s3file_composite_pk/migration.sql	Migration switching S3File to composite PK by dropping/recreating the table.
backend/package.json	Adds AWS SDK v3 S3 client and presigner dependencies.
backend/.env.example	Documents S3 configuration (currently still describes browser-direct presigned PUT flow).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[OhYee]

Superseded — folded into #163 to avoid the add-then-delete churn between the frontend signed-URL flow and the backend-direct refactor. The combined PR has the same final state with cleaner per-commit logic.