Skip to content

Conversation

@tan-z-tan
Copy link

WHY

Firestore has a strict document size limit of 1MB. When the Yjs document state exceeds this limit, write operations fail, preventing users from saving large documents. We need a mechanism to handle documents larger than 1MB.

WHAT

  • Implemented Chunking for Large Documents:
    • Modified saveToFirestore to check if the encoded Yjs document size exceeds 800KB.
    • If the size limit is exceeded, the document is split into multiple chunks.
    • These chunks are saved as individual documents in a yfire_chunks subcollection under the main document.
    • The main document is updated with a flag chunked: true and the content field is cleared.
  • Updated Loading Logic:
    • Modified trackData to handle chunked: true documents.
    • It now fetches all chunks from the yfire_chunks subcollection, reassembles them in order, and applies the update to the Yjs document.
  • Removed GCS Logic:
    • Removed the incomplete and unused Google Cloud Storage (GCS) integration code to keep the provider focused and clean.

@tan-z-tan
Copy link
Author

@deathg0d Hi.
This PR was originally intended for my fork, but if you find it useful, I’d be happy if you consider merging it. If not, feel free to close it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant