You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: modules/ROOT/pages/server-compatibility-buckets.adoc
+5-1Lines changed: 5 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -52,7 +52,11 @@ For more information, see xref:server:learn:data/expiration.adoc[].
52
52
Sync Gateway does not support Bucket-level TTL, make sure your buckets have their `maxTTL` setting set to `0`.
53
53
If the bucket setting has a non-zero `maxTTL` value set, Sync Gateway returns an error prompting you to set the value to `0` in the Couchbase Server Admin UI.
54
54
55
-
NOTE: You can still set Document expiration settings and Collection maxTTL.
55
+
Similarly, do not set Collection-level TTL (`maxTTL` on collections) as this can interfere with Sync Gateway's internal documents, including those with `_sync` prefixes and other system documents that are essential for proper operation.
56
+
If these system documents expire due to collection-level TTL, Sync Gateway may malfunction or fail to operate properly.
57
+
You can use per-collection sync functions to set expiry on all documents within a collection when you need TTL-like behavior at the collection level, while preserving Sync Gateway's system documents.
58
+
59
+
NOTE: You can still set Document expiration settings on individual documents.
Copy file name to clipboardExpand all lines: modules/ROOT/pages/sync-inter-syncgateway-overview.adoc
+29-5Lines changed: 29 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -58,14 +58,15 @@ A typical architecture for this use cases is shown in <<icr-cloud-to-edge>>
58
58
image::icr-cloud-to-edge200712.svg[,600]
59
59
60
60
=== Active-to-active Mobile Synchronization
61
-
Here edge clusters containing active Sync{nbsp}Gateway nodes are replicated between geographically separate cloud-based Sync Gateway deployments.
62
-
An ideal use-case for inter-Sync{nbsp}Gateway replication, which was designed to keep clusters in different data centers in sync.
63
61
64
-
The Couchbase Server Cross Data Center Replication API (xref:server:manage:manage-xdcr/xdcr-management-overview.adoc[XDCR]) is similarly used to replicate between Couchbase Server clusters.
65
-
*However, the inter-Sync{nbsp}Gateway replication feature is specifically designed for Sync Gateway deployments and it must be used for replication between mobile clusters.* {fnxdcr}
62
+
Inter-Sync Gateway replication replicates edge clusters containing Sync{nbsp}Gateway nodes between geographically separate cloud-based Sync Gateway deployments.
63
+
This provides an ideal use-case for inter-Sync{nbsp}Gateway replication, which was designed to keep clusters in different data centers in sync.
66
64
65
+
Couchbase Server uses the Cross Data Center Replication API (xref:server:manage:manage-xdcr/xdcr-management-overview.adoc[XDCR]) similarly to replicate between Couchbase Server clusters.
66
+
*However, inter-Sync{nbsp}Gateway replication feature is specifically designed for Sync Gateway deployments and must be used for replication between mobile clusters.* {fnxdcr}
67
67
68
-
A typical architecture for this use case is shown in <<icr-active-mobile>>
68
+
69
+
<<icr-active-mobile>> shows a typical architecture for this use case.
69
70
70
71
[[icr-active-mobile]]
71
72
.Active-to-active mobile synchronization
@@ -148,6 +149,29 @@ NOTE: Push replications to pre-2.8 targets do not use Delta Sync
Inter-Sync Gateway replication supports named collections, including:
155
+
156
+
* Enabling or disabling collections replication.
157
+
* Limit replication to specific collections.
158
+
* Map local collections to differently named remote collections.
159
+
160
+
For detailed configuration of collections parameters (`collections_enabled`, `collections_local`, `collections_remote`), see xref:configuration-schema-isgr.adoc#replication[Run Inter-Sync Gateway Replication Configuration Schema].
161
+
162
+
==== Collection Mapping
163
+
164
+
Collection mapping allows you to replicate local collections to differently named collections on the remote target. You configure this using two parallel arrays:
165
+
166
+
* `collections_local` - specifies the collections to replicate from the source.
167
+
* `collections_remote` - specifies the corresponding target collection names on the remote.
168
+
169
+
The mapping works positionally, where the first collection in `collections_local` maps to the first collection in `collections_remote`, the second to the second, and so on.
170
+
171
+
NOTE: You must verify all specified collections exist in both source and target Sync Gateway database configurations before starting replication.
172
+
Sync Gateway does not automatically create missing collections or sync new collections after replication begins.
173
+
174
+
This collection mapping capability adapts the mapping concept from XDCR for Sync Gateway deployments. For information about xref:server:learn:clusters-and-availability/xdcr-with-scopes-and-collections.adoc#implicit-mapping[Implicit Mapping] and xref:server:learn:clusters-and-availability/xdcr-with-scopes-and-collections.adoc#explicit-mapping[Explicit Mapping] in the XDCR documentation.
:fn-sgw: footnote:fn-sgw[Prior to Release 2.5, Server 5.0 all writes had to go through Sync Gateway, or had to use bucket shadowing to ensure that the security and replication metadata needed by mobile applications was preserved.]
{sgw-t} will automatically sync database changes if {configuration-schema-database--xref--import-docs} and {configuration-schema-database--xref--enable-shared-bucket-access} are set 'true'.
32
+
{sgw-te} will automatically sync database changes if {configuration-schema-database--xref--import-docs} and {configuration-schema-database--xref--enable-shared-bucket-access} are set 'true'.
34
33
35
-
enable_shared_bucket_access::
36
-
This setting ensures that both {sgw-t} and {cbs-t} can read and write to the same bucket simultaneously and that {sgw-t} can access {cbs} document XATTRS +
37
-
See: <<sgw-paths>> and {configuration-schema-database--xref--enable-shared-bucket-access}.
34
+
* Enable_Shared_Bucket_Access
38
35
+
39
-
Setting this property to `true` will be required in a future release, and `false` will not be supported.
36
+
This setting ensures that both {sgw} and {cbs-te} can read and write to the same bucket simultaneously and that {sgw} can access {cbs} document XATTRS +
37
+
See: <<sgw-paths>> and {configuration-schema-database--xref--enable-shared-bucket-access}.
40
38
41
-
import_docs::
42
-
Setting this property true ensures that the {sgw-t} node performs import processing, obtaining the mobile metadata it requires to replicate changes -- see: <<import-process>>.
39
+
* Import Docs
40
+
+
41
+
Setting this property true ensures that the {sgw} node performs import processing, obtaining the mobile metadata it requires to replicate changes -- see: <<import-process>>.
43
42
44
43
You can configure both these properties using the Admin Rest API {configuration-schema-database--xref} endpoint.
Mobile applications require additional metadata in order to manage security and replication.
56
55
57
-
{sgw-t} stores this information using {cbs} XATTRs (see {cbs} documentation on {server-data-xattr-fundamentals--xref}).
56
+
{sgw} stores this information using {cbs} XATTRs (see {cbs} documentation on {server-data-xattr-fundamentals--xref}).
58
57
59
58
60
59
=== Extended Attributes (XATTRs)
@@ -72,24 +71,24 @@ Extended attributes:
72
71
* Can be accessed via {cbs} SDKs using the sub-document API, via command-line tools, and via views.
73
72
74
73
* Are accessible from {sqlpp} in {cbs} using the `().xattrs` property. +
75
-
For example, `SELECT meta().xattrs._sync from travel-sample where Meta().id = "user::demo";`.
74
+
For example, `SELECT meta().xattrs._sync from travel-sample where Meta().id = "user::demo"`;.
76
75
77
76
78
-
Both {sgw-t} and {cbs} use a System extended attribute, with the following characteristics to support mobile convergence (shared bucket access):
77
+
Both {sgw} and {cbs} use a System extended attribute, with the following characteristics to support mobile convergence (shared bucket access):
79
78
80
79
* Shares lifetime with the document metadata - when a document is deleted, system xattrs are preserved with the tombstone.
81
80
* Allocated 1MB of storage, independent of the 20MB available for the document
82
81
83
-
WARNING: The sync metadata is maintained internally by {sgw-t} and its structure can change at any time.
82
+
WARNING: The sync metadata is maintained internally by {sgw} and its structure can change at any time.
84
83
It should not be used to drive business logic of applications.
85
84
The direct use of the {sqlpp} query is unsupported and must not be used in production environments.
86
-
The `_raw` endpoint (/db/_raw/{docid}) on {sgw-t}'s Admin REST API returns both the document and its associated mobile metadata.
85
+
The `_raw` endpoint (/db/_raw/{docid}) on {sgw}'s Admin REST API returns both the document and its associated mobile metadata.
87
86
88
87
=== Documents
89
88
90
-
With bucket-sharing enabled, {cbs} documents can be inserted directly (using {sqlpp} or SDKs) or by using {sgw-t}'s {rest-api--xref}.
89
+
With bucket-sharing enabled, {cbs} documents can be inserted directly (using _{sqlpp}_ or _SDKs_) or by using {sgw}'s {rest-api--xref}.
91
90
92
-
{sgw-t} {fn-sgw1x5} creates the metadata it needs by abstracting it from the SDK or {sqlpp} applications reading and writing data directly to {cbs} buckets.
91
+
{sgw} {fn-sgw1x5} creates the metadata it needs by abstracting it from the SDK or {sqlpp} applications reading and writing data directly to {cbs} buckets.
93
92
It uses {cbs} XATTRs {fn-cbs5x0} to store that metadata into an external document fragment -- see {xref-svr-pg-xattrs}.
94
93
95
94
// The REST API will also include the following behavioral changes:
@@ -100,7 +99,7 @@ It uses {cbs} XATTRs {fn-cbs5x0} to store that metadata into an external documen
100
99
=== Blobs and Attachments
101
100
102
101
.Couchbase Server SDK/{sqlpp}
103
-
Use {sgw-t}'s REST API's {xref-sgw-ep-public-api-attachment} endpoints to
102
+
Use {sgw}'s REST API's {xref-sgw-ep-public-api-attachment} endpoints to
104
103
manage attachments and blob data; you cannot use {cbs} SDKs to do this
105
104
directly.
106
105
@@ -115,7 +114,7 @@ Couchbase Lite apps seamlessly handle blobs and attachments, see the appropriate
115
114
include::partial$blocklinks-cbl.adoc[]
116
115
117
116
.Using a WebApp
118
-
Attachments can be accessed through {sgw-t}'s REST API using the xref:rest_api_public.adoc#tag/Document-Attachment/operation/get_keyspace-docid-attach[+/{keyspace}/{docid}/{attach}+] endpoint.
117
+
Attachments can be accessed through {sgw}'s REST API using the xref:rest_api_public.adoc#tag/Document-Attachment/operation/get_keyspace-docid-attach[+/{keyspace}/{docid}/{attach}+] endpoint.
119
118
120
119
=== Tombstone Revisions
121
120
@@ -130,21 +129,23 @@ As stated, mobile metadata is not kept in the document, but in a system extended
130
129
The {sqlpp} query language {fnref-cbs5x0} supports the ability to query these extended attributes (XATTRS) and hence the document's sync metadata -- see: <<simple-query>>.
131
130
132
131
[#simple-query]
133
-
.Querying XATTRS-bsed sync metadata
132
+
.Querying XATTRS-based sync metadata
134
133
====
135
-
[source,sqlpp]
134
+
[source,sql]
136
135
----
137
136
SELECT meta().xattrs._sync FROM scope.collection WHERE meta().id = "mydocId"
138
137
----
139
138
====
140
139
141
-
WARNING: The sync metadata is maintained internally by {sgw-t} and its structure can change at any time.
142
-
It should not be used to drive business logic of applications. The direct use of the {sqlpp} query is *unsupported* and must not be used in production environments.
140
+
WARNING: {sgw} maintains the sync metadata internally, and its structure can change at any time.
141
+
Applications must not use it for business logic.
142
+
The direct use of the {sqlpp} query or modifying the internal sync metadata contents to drive the business logic is unsupported and must not be used in production environments.
143
+
The sync metadata includes the `_sync` extended attribute (XATTR) in use case documents and all `_sync:` prefixed documents in Sync Gateway connected Buckets.
143
144
144
145
=== Enable Shared Bucket Access
145
146
146
147
Shared bucket access is an opt-in feature.
147
-
You can enable it without bringing down the entire {sgw-t} cluster -- see <<enable-sba>>.
148
+
You can enable it without bringing down the entire {sgw} cluster -- see <<enable-sba>>.
148
149
149
150
[#enable-sba]
150
151
.Enable Bucket-Sharing
@@ -161,7 +162,7 @@ You can enable it without bringing down the entire {sgw-t} cluster -- see <<ena
161
162
}
162
163
}
163
164
----
164
-
<.> The `import_docs` property is used to specify that a {sgw-t} node participates (exclusively) in {import-process--xref}. The mechanism by which {sgw-t} incorporates changes to data buckets it shares with {cbs} -- see: <<import-process>>.
165
+
<.> The `import_docs` property is used to specify that a {sgw} node participates (exclusively) in {import-process--xref}. The mechanism by which {sgw} incorporates changes to data buckets it shares with {cbs} -- see: <<import-process>>.
165
166
166
167
====
167
168
@@ -171,37 +172,30 @@ You can enable it without bringing down the entire {sgw-t} cluster -- see <<ena
171
172
172
173
173
174
The *import process* is a key part of mobile convergence.
174
-
It is the means by which {sgw-t} becomes aware of non-{sgw-t} data changes and obtains the mobile metadata it requires to replicate changes.
175
+
It is the means by which {sgw} becomes aware of non-{sgw} data changes and obtains the mobile metadata it requires to replicate changes.
175
176
176
177
image::shared-bucket-access.png[]
177
178
178
-
Any non-{sgw-t} change is eligible for import.
179
+
Any non-{sgw} change is eligible for import.
179
180
The document is first run through the Sync Function to compute read security and routing, with the following differences:
180
181
181
-
* The import is processed with an admin user context in the Sync Function, similar to writes made through the {sgw-t} Admin API.
182
+
* The import is processed with an admin user context in the Sync Function, similar to writes made through the {sgw} Admin API.
182
183
This means that `requireAccess`, `requireUser` and `requireRole` calls in the Sync Function are treated as no-ops.
183
184
* During import, `oldDoc` is `nil` when the Sync Function is executed.
184
185
185
-
You can specify a filter function using the {configuration-schema-database--pfx}#import_filter[import_filter] property, which will only import specific documents.
186
+
You can specify a filter function using the {configuration-schema-database--pfx--db}_import_filter[import_filter] property, which will only import specific documents.
186
187
187
188
TIP: Use the {bootstrap-schema--xref--logging-console-log-keys} log key to troubleshoot import processing issues in the logs.
188
189
189
190
=== Configuration
190
191
191
192
Note that `import_docs` only takes effect if the `enabled_shared_bucket_access` is set to `true`.
192
193
193
-
****
194
-
[.edition]#{enterprise}#
195
-
194
+
{enterprise}::
196
195
The `import_docs` parameter defaults to `true`, implying that, by default, all nodes in a cluster participate in import processing.
197
196
To exclude a node, set `"import_docs": false`.
198
-
199
-
'''
200
-
201
-
[.edition]#{community}#
202
-
203
-
The `import_docs` parameter defaults to `false` and must be explicitly set to `true`.
204
-
****
197
+
{community}::
198
+
The `import_docs` parameter defaults to false and must be explicitly set to `true`.
205
199
206
200
The following table describes the key behavior differences between Community Edition and Enterprise Edition when `import_docs` is enabled, disabled or not set at all.
207
201
@@ -241,7 +235,7 @@ The following table describes the key behavior differences between Community Edi
241
235
|===
242
236
243
237
=== High Availability
244
-
In _Enterprise Edition_, import processing work is sharded across all {sgw-t} nodes with import enabled.
238
+
In _Enterprise Edition_, import processing work is sharded across all {sgw} nodes with import enabled.
245
239
This implies that if one of the nodes fail, the failed shard is automatically picked up by the remaining nodes in the cluster.
246
240
This way, you get High Availability of import processing.
247
241
@@ -254,7 +248,7 @@ As described in the table above, if `import_docs` is set to `false`, the node wi
254
248
This configuration is specifically recommended for workload isolation: to isolate import nodes from the client-facing nodes.
255
249
Workload isolation may be preferable in deployments with high write throughput.
256
250
257
-
The following diagram shows an example architecture of two {sgw-t} nodes handling the incoming client connections (`import_docs: false`) and two nodes sharing the import processing (`import_docs: true`).
251
+
The following diagram shows an example architecture of two {sgw} nodes handling the incoming client connections (`import_docs: false`) and two nodes sharing the import processing (`import_docs: true`).
* Check out our getting started tutorial for more on how to setup, configure and run {sgw-t} replications - xref:tutorials:userprofile-sync:userprofile_sync.adoc[Sync tutorial]
267
+
* Check out our getting started tutorial for more on how to setup, configure and run {sgw} replications - xref:tutorials:userprofile-sync:userprofile_sync.adoc[Sync tutorial]
274
268
275
269
* Further reading:
276
270
** {cbs} documentation on {server-data-xattr-fundamentals--xref}
277
271
278
272
** Configuration file references:
279
273
280
274
*** {enable-sba-config-item} to enable convergence for a given database.
281
-
*** {configuration-schema-database--pfx}#import_docs[$dbname.import_docs] to give a particular {sgw-t} node the role of importing the documents.
282
-
*** {configuration-schema-database--pfx}#import_filter[$dbname.import_filter] to select which document(s) to make aware to mobile clients.
275
+
*** {configuration-schema-database--pfx--db}-import_docs[$dbname.import_docs] to give a particular {sgw} node the role of importing the documents.
276
+
*** {configuration-schema-database--pfx--db}-import_filter[$dbname.import_filter] to select which document(s) to make aware to mobile clients.
0 commit comments