MongoDB Sharded Cluster Support

[stevensJourney]

Overview

This adds support for sharded MongoDB replication connections.

The checks for if a sharded connection has been provided have been updated to now allow sharded connections.

The lastWrite?.majorityOpTime is not available in sharded clusters, but it seems like we could use the clusterTime for the initial snapshot in this case.

This has been tested locally with a MongoDB Atlas sharded cluster. Testing included:

• ✅ Initial replication
• ✅ Streaming changes (from MongoDB Atlas and connected clients) after initial replication
• ✅ Stopping the service, making changes in Atlas and restarting the service. Verifying that changes made when the service was stopped were streamed correctly.
• ⚠️ Dropping a collection which contains items. This test might have uncovered an existing bug. The table contents were not cleared for clients until another change was made on the Atlas DB. This might be due to drop events not progressing the checkpoint.
• ✅ Dropping the source DB. This correctly caused the sync rules to reprocess.
• ❌ The test for checking if msg == 'isdbgrid was also used to guard against unsupported Atlas Serverless instances. I tested an Atlas Flex Tier DB (which is meant to be the replacement for Serverless) and noticed that making changes on Atlas while the service has been stopped and then resuming the service would cause all data to be cleared from clients which have already synced. As a side note: interestingly when connecting this flex instance the msg value is undefined and not isdbgrid. We may need some other check to guard against these instances.
• ❌ The unit tests fail when linked to a sharded cluster. Some tests show concerning failures in the expected bucket storage values.

[changeset-bot]

🦋 Changeset detected

Latest commit: cdf29ca

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 9 packages

Name	Type
@powersync/service-module-mongodb	Minor
@powersync/service-core	Minor
@powersync/service-image	Minor
@powersync/service-core-tests	Patch
@powersync/service-module-mongodb-storage	Patch
@powersync/service-module-mysql	Patch
@powersync/service-module-postgres-storage	Patch
@powersync/service-module-postgres	Patch
test-client	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR