Skip to content

Deduplicate CAD assets and stop re-fingerprinting undo history during .skf autosave - #101

Open
davidsteinmeyer wants to merge 1 commit into
Formsmith746:mainfrom
davidsteinmeyer:ds_freeze_bug
Open

Deduplicate CAD assets and stop re-fingerprinting undo history during .skf autosave#101
davidsteinmeyer wants to merge 1 commit into
Formsmith746:mainfrom
davidsteinmeyer:ds_freeze_bug

Conversation

@davidsteinmeyer

Copy link
Copy Markdown

Summary

Autosave generates a full .skf archive containing the complete undo history. Export was encoding and SHA-256-hashing every resource in every history state, deduplicating only after hashing, inlining the large cadDisplayEdges cache into every node of every state, and re-fingerprinting the whole history twice. On a real 6.9 MB project this produced a 30.8 MB project.json and a ~3.8 s export on the main thread. This PR makes asset handling content-addressed before the expensive work, stores display edges once as a shared asset, and reuses history fingerprints. History behavior is fully preserved.

Root cause

  • SkfArchiveBuilder.addAsset computed sha256Hex(bytes) before checking its kind:sha256 dedup map, so identical resources were re-encoded and re-hashed once per state. addDerivedMesh cached only by object identity, and cadBrep, brepStep, and image data URLs had no pre-hash cache at all.
  • cadDisplayEdges was left in the node definition and inlined per node per state. In the real project 230 occurrences held only 2 unique payloads (0.21 MB unique -> 24.5 MB in project.json).
  • canonicalizeShape always allocated new nested arrays, so the fingerprint WeakMap missed, and exportSkfProject fingerprinted the entire history twice.

Changes

  • Added recordByResourceKey and addEncodedAsset(kind, resourceKey, encode, options) to SkfArchiveBuilder; it returns a cached promise on a key hit and only invokes the encode() thunk on a miss, so encoding and hashing are skipped for repeated content.
  • addDerivedMesh now keys on mesh.storageResourceId ?? immutableResourceFingerprint(mesh) (content-based) instead of object identity.
  • serializeShapeNode routes imagePlate, sketchProfile.images, brepStep, and cadBrep through addEncodedAsset with content keys.
  • cadDisplayEdges is now extracted from the node definition, stored once in the asset table (kind brep, media type application/vnd.sketchforge.display-edges), and referenced by the new optional cadDisplayEdgesAssetId node field. The importer restores it and also shares one array per distinct payload for legacy inline packages.
  • exportSkfProject reuses the fingerprint already computed by hydrateEditorHistoryState and only re-fingerprints entries whose shapes were actually repaired.
  • repairDuplicateGroupedObjectIds returns the original array when nothing changed.
  • canonicalizeShape is now identity-preserving when a shape is already canonical, so resource fingerprints cached by object identity survive repeated canonicalization.

Why this fixes the issue

  • Hashing drops from O(history_states x resources) to O(unique_resources); on a 200-state shared-resource test, digest calls fall from 201 to 2.
  • cadDisplayEdges storage drops from 24.5 MB to 0.21 MB, shrinking the real project's project.json from 30.8 MB to ~6.5 MB and the archive from 7.8 MB to 1.9 MB.
  • History fingerprinting is now cached and single-pass.

Testing

  • npm run typecheck
  • Manually tested the affected workflow

Notes

New unit tests:

  • "encodes and hashes each unique history resource once instead of once per undo state"
  • "deduplicates large display-edge caches across undo states"

Performance impact

  • Real project autosave: ~3.8 s -> ~1.0 s; archive 7.8 MB -> 1.9 MB.
  • History fingerprinting: ~1.4 s -> ~0.1 s.
  • Shared-resource synthetic case (200 states): 201 -> 2 crypto digests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant