| description |
|---|
Bi-directional Data Sync with MongoDB - on-premise and to the cloud |
ObjectBox Data Sync syncs data with MongoDB using the integrated MongoDB Sync Connector. Changes made on ObjectBox clients are synchronized in real-time to MongoDB and vice versa.
{% hint style="info" %} Get your MongoDB Sync Connector Alpha {% endhint %}
ObjectBox Sync Connector for MongoDB: Architecture
ObjectBox Sync brings your data in MongoDB to the edge (e.g. mobile and IoT devices, big and small servers) and synchronizes changes back to MongoDB. By using ObjectBox Sync, you can make your MongoDB data always available: continue to work offline and sync in real-time when online.
If you are coming from Atlas Device Sync, you already now that it reaches its end-of-life on September 30, 2025. Thus, it's a good idea to contact ObjectBox asap to get started with your migration.
While ObjectBox Sync is an established product, the MongoDB Connector is in its early stages. You can still get started with POCs or integrations right away. The timeline below enables you to align your development with ours and pick up speed. Once new features are out, you profit immediately.
| Planned Date | Released | Connector Version | Main features |
|---|---|---|---|
| Oktober 2024 | ✅ | Alpha 1 | Two-way sync between MongoDB and ObjectBox for changes that are happening live. |
| January 2025 | ✅ | Alpha 2 | JWT authentication, Mapping for all MongoDB data types |
| February 2025 | ✅ | Alpha 3 | Relation mapping (to-one and many-to-many) |
| March 2025 | ✅ | Alpha 4 | Support for additional MongoDB ID types |
| March 2025 | Beta 1 | Import initial data from MongoDB, pick-up sync at any point | |
| April 2025 | Beta 2 | Server-side rules to select data for sync per user | |
| Mai 2025 | 1.0 | Initial release incorporating most important user feedback |
We'll update the plan regularly.
{% hint style="warning" %} Do not connect to your MongoDB production instance! Use a separate instance for testing purposes. {% endhint %}
Setting up Sync server and its MongoDB Sync Connector involves the following steps:
- Create and provide a Data Model to Sync Server using a model JSON file.
- Run and test Sync Server without connecting to MongoDB and validate it actually syncs data.
- Ensure that your MongoDB instance is a replica set. This is required for the MongoDB Sync Connector to work.
- Configure the MongoDB connection and run Sync Server. E.g. provide the connection URL to the Sync Server and restart it. This can be done via CLI arguments or the JSON configuration file.
- Verify the MongoDB connection using the Admin UI.
Read on for details.
In general, the ObjectBox Sync server requires a data model to be provided (a JSON file, see objectbox-sync-server.md). This data model is also used by the MongoDB Sync Connector to map data between ObjectBox and MongoDB. On how this works, see the chapter on data mapping below.
To avoid any later issues, run and test Sync Server without connecting to MongoDB and your client application and validate data is synced.
See the objectbox-sync-server.md page on how to run Sync Server.
{% hint style="info" %} MongoDB Atlas clusters are already replica sets, no additional configuration is required. {% endhint %}
Only a MongoDB replica set instance provides the necessary features for the MongoDB Sync Connector to work (e.g. MongoDB's change streams).
A local standalone MongoDB instance (MongoDB Community Edition is fine) can be converted to a replica set. You can do this either by following the official MongoDB documentation, or by following these simplified steps (tested on Ubuntu Linux) for a single node setup:
-
Stop the MongoDB service:
sudo systemctl stop mongod -
Edit the MongoDB configuration file:
sudo vi /etc/mongod.conf -
Add the following lines to the configuration file:
replication: replSetName: "rs0"
-
Start the MongoDB service:
sudo systemctl start mongod -
Connect to the MongoDB shell:
mongosh -
Initialize the replica set via the MongoDB shell:
rs.initiate()
To configure the ObjectBox MongoDB Sync Connector via CLI arguments when starting Sync Server (see objectbox-sync-server.md), you can use the following options:
--mongo-url: The MongoDB connection string (URL or URI). This can be an empty string for the default127.0.0.1:27017host.--mongo-db: The primary MongoDB database name; the "database" containing the collections used for sync. By default this is "objectbox_sync".
Alternatively, configure the MongoDB connection in the Sync Server configuration file (see sync-server-configuration). In your sync-server-config.json, add a new mongoDb node which contains key/value pairs for MongoDB specific configuration attributes:
{
...
"mongoDb": {
"url": "1.2.3.4",
"database": "db123"
}
}{% hint style="warning" %}
If your Sync server version's date is lower than 2024-10-07, use {"mongoUrl": "1.2.3.4"} without the mongoDb config node.
{% endhint %}
Use the ObjectBox Sync Server Admin web app to verify the MongoDB connection works.
ObjectBox and MongoDB have many similarities. Nevertheless, it's important to understand the differences in terminology and concepts between the two databases. The following table illustrates these differences. It serves as background information on how to map things between the two systems:
| Concept | ObjectBox | MongoDB |
|---|---|---|
| Database containing the data | A store, grouped into types (data classes). | A database, grouped into collections. |
| What your application "opens" | Using a name or directory a single store (database) is opened locally on device. (In the case of Sync, this is the "client database".) Multiple stores can be opened, which are strictly separate. | A client is used to connect to a MongoDB server. All databases can be accessed on the server remotely. |
| Arranging data in a database (e.g. data sets) | A box holds all objects of the same type. A type typically matches a data class in programming languages. It's part of a strict schema (data model) that is enforced. The type definition consists of a fixed set of properties. | Collections are used to group documents together. By default, no strict rules are imposed (schema-less). |
| Data record | An object is an instance of a type. It can have data values for the properties defined in the type. | A document is a set of data values called fields. It's very similar to a JSON file. |
| Modelling related data | Objects can have to-one and to-many relationships to other objects. Relationships are bidirectional. Data is typically normalized. | Typically, a document embeds all "related data" into itself resulting in larger documents. Data is typically not normalized. Alternatively, one can also choose to use references, which is more similar to relationships. |
| Many-to-many relationships | Fully supported. Unlike to-one relationships, the data is stored outside the object. Updating relations is very efficient. There's no user-defined order and no duplicates. | Alternative modelling by embedding or referencing documents. Inside the document, you can have an array of IDs in a user-defined order allowing duplicates. |
| Nested data records | Nested data is supported by flex properties. E.g. these allow maps, which can contain nested data structures (JSON-like). | Documents contain nested data by default. |
| Identifiers (IDs) | IDs are 64-bit integers. They are unique within its box and database instance. | MongoDB uses Object ID (short: OID; 12 bytes) by default and supports other ID types. OIDs are unique within their collection and likely globally unique. |
The data model used by ObjectBox defines types, which are mapped to MongoDB collections. Similarly, the properties of a type are mapped to keys inside a MongoDB document. Thus, you should ensure that the ObjectBox data model matches the MongoDB schema. E.g. if you have an existing MongoDB database, ensure to match the names when you create the ObjectBox model.
When you compare the data structure of simple data elements, the difference is not big. Without nested data, the main difference you will note is the ID type.
.png)
Note: nested documents are supported via the ObjectBox "Flex" property type, which can hold a map-like (JSON-like) structure. We are also considering alternatives to this, so let us know if you have specific requirements.
ObjectBox Sync automatically maps IDs when syncing with an external system like MongoDB. This way, you can use the native IDs in each system, 64-bit integer IDs in ObjectBox and e.g. 12-byte object IDs in MongoID. This also means that MongoDB ID is not present in ObjectBox objects and vice versa.
{% hint style="warning" %}
ObjectBox IDs are only valid on their local device. Do not store them manually (apart from relations) e.g. as a custom list or vector, when you want to sync to other devices.
For details, you can refer to the internal ID mapping docs that happens on each ObjectBox device.
{% endhint %}
Besides the Object ID, ObjectBox supports most common ID types offered by MongoDB. IDs of incoming documents from MongoDB are automatically detected and mapped to ObjectBox local IDs. This mapping is persisted and thus any change made on the ObjectBox side can be mapped back to the initial ID type and value.
For newly created (inserted) objects on the ObjectBox side, a new MongoDB object ID (OID) is created by default. You can customize the MongoDB ID types in the ObjectBox data model: for the ID property, define a "external property type" on the ID property. E.g. a definition of the ID field in the ObjectBox entities would look similar like this pseudo code (syntax varies according to programming language):
@Id @ExternalType(UUID) int64 id;
The following table shows the supported ID types:
| MongoDB type | Incoming from MongoDB | IDs for new documents created in ObjectBox |
|---|---|---|
| Object ID | ✅ | ✅ This is the default type |
| UUID (Binary with UUID subtype) | ✅ | ✅ External types: Uuid (V7) or UuidV4 |
| String | ✅ | ✅ External types: UuidString (V7) or UuidV4String |
| Binary | ✅ | Uses default MongoDB Object Id |
| Int64 | ✅ | Uses default MongoDB Object Id |
| Int32 | ✅ | Uses default MongoDB Object Id |
{% hint style="info" %} If you want to learn more about ObjectBox relations, check the relation documentation. {% endhint %}
ObjectBox Sync also happens automatically maps IDs used in relations (starting with version Alpha 3 of MongoDB Sync Connector).
Consider ObjectBox to-one relations, which have a single relation property pointing to another object using a 64-bit integer ID. This becomes a reference field in MongoDB's document containing the MongoDB object ID (OID). See the following illustration for an example:
.png)
To-One Relationship Example: motherId
The supported ID types also apply for relations, e.g. a "Person" document would have an UUID as the _id field value, relations to it would also use the UUID as the relation ID on the MongoDB side.
Many-to-many relations work a bit different. As illustrated in the table above, many-to-many relations work differently in ObjectBox and MongoDB. For mapping between the following rules apply:
- On the MongoDB side, many-to-many relations are stored as an array of ID references:
- The IDs are stored inside the document "owning" the relationship.
- The owning side of a relationship is always the same type (collection).
- If you want to, you can make this relation bidirectional, by adding IDs to the "target side" of the relationship. Just do not make this visible in the ObjectBox data model.
- On the ObjectBox side, its native many-to-many relationships are used:
- They are bidirectional, e.g. you can define it on the owning and target side.
- They can be updated efficiently without touching the object
- They can be used in queries to link types (aka join).
As to-many relations consist of ID values, all supported types can be used. In theory, different ID types can be used in the same to-many, relation. However, it's usually good practice to stick to a single ID type per MongoDB collection if possible.