DOC-12777: Update Eventing REST API with sync-gateway-aware options (#163)

* Update eventing schemas to master
* Add examples for Sync Gateway compatibility
* Updates after review: add overlay for new settings
* Updates after review: update Gradle to normalize spec and apply overlay
* Updates after review: add cursor limit
* Rebuild output documentation

Author: simon-dew

docs/modules/eventing-rest-api/pages/index.adoc

@@ -4276,8 +4276,7 @@ GET /api/v1/config
 [markdown]
 --
 Shows all global configuration settings.
-There are currently just two settings: `enable_debugger` and `ram_quota`.
-Both of these settings can also be adjusted via the UI.
+Note that the `enable_debugger` and `ram_quota` settings can also be adjusted via the UI.
 --
 
 
@@ -7846,6 +7845,38 @@ maximum time the handler can run before it is forcefully terminated (in seconds)
 a¦ Integer
 
 
+a¦ 
+*cursor_checkpoint_timeout* +
+_optional_
+a¦ 
+
+[markdown]
+--
+The maximum time the checkpoint writer can run before it is forcefully terminated (in seconds).
+--
+
+[%hardbreaks]
+*Minimum:* `1`
+{blank}
+a¦ Integer
+
+
+a¦ 
+*on_deploy_timeout* +
+_optional_
+a¦ 
+
+[markdown]
+--
+maximum time the OnDeploy handler can run before it is terminated (in seconds)
+--
+
+[%hardbreaks]
+*Minimum:* `1`
+{blank}
+a¦ Integer
+
+
 a¦ 
 *language_compatibility* +
 _optional_
@@ -8223,6 +8254,138 @@ maximum allowable curl call response in 'MegaBytes'. Setting the value to 0 lift
 a¦ Integer
 
 
+a¦ 
+*allow_transaction_mutations* +
+_optional_
+a¦ 
+
+[markdown]
+--
+allow staged transaction mutations
+--
+
+[%hardbreaks]
+{blank}
+a¦ Boolean
+
+
+a¦ 
+*allow_sync_documents* +
+_optional_
+a¦ 
+
+[markdown]
+--
+Specifies whether the function allows Sync Gateway mutations.
+
+* By default, this setting is `true`, for compatibility with previous versions of Couchbase Server.
+
+* When this setting is `false`, the specified function skips all internal Sync Gateway documents, whose IDs are prefixed with `_sync`.
+This enables the function to work with Sync Gateway.
+
+You must ensure that none of the documents which contain your own working data have IDs which are prefixed with `_sync`.
+(Note that internal Sync Gateway attachment documents, whose IDs are prefixed with `_sync:att`, are still processed by the specified function.)
+--
+
+[%hardbreaks]
+{blank}
+a¦ Boolean
+
+
+a¦ 
+*cursor_aware* +
+_optional_
+a¦ 
+
+[markdown]
+--
+Specifies whether the function suppresses potential duplicate mutations caused by App Services or Sync Gateway book-keeping.
+
+Enabling this setting guarantees that the Eventing function will only trigger once for any given mutation received from App Services or Sync Gateway.
+
+Note that enabling this setting may have a noticeable impact on the performance of the Eventing function.
+--
+
+[%hardbreaks]
+{blank}
+a¦ Boolean
+
+
+a¦ 
+*high_seq_check_interval* +
+_optional_
+a¦ 
+
+[markdown]
+--
+number of milliseconds before checking for high seq number
+--
+
+[%hardbreaks]
+{blank}
+a¦ Integer
+
+
+a¦ 
+*max_unacked_bytes* +
+_optional_
+a¦ 
+
+[markdown]
+--
+max MBs to wait to send more bytes to c++ side
+--
+
+[%hardbreaks]
+{blank}
+a¦ Integer
+
+
+a¦ 
+*max_unacked_count* +
+_optional_
+a¦ 
+
+[markdown]
+--
+max number of messages on c++ side
+--
+
+[%hardbreaks]
+{blank}
+a¦ Integer
+
+
+a¦ 
+*message_flush_time* +
+_optional_
+a¦ 
+
+[markdown]
+--
+number of milliseconds before sending message to c++ side
+--
+
+[%hardbreaks]
+{blank}
+a¦ Integer
+
+
+a¦ 
+*max_parallel_vb* +
+_optional_
+a¦ 
+
+[markdown]
+--
+number of parallel vb request per cpp thread
+--
+
+[%hardbreaks]
+{blank}
+a¦ Integer
+
+
 |===
 
 //end::settings_schema[]
@@ -8298,6 +8461,28 @@ For details, see [Debugging and Diagnosability](/server/8.0/eventing/eventing-de
 a¦ Boolean
 
 
+a¦ 
+*cursor_limit* +
+_optional_
+a¦ 
+
+[markdown]
+--
+The maximum number of cursor-aware Eventing functions that can coexist on a given source keyspace. (A cursor-aware Eventing function is one for which the `cursor_aware` setting is `true`.)
+
+Increasing this setting enables more cursor-aware Eventing functions to register and listen to any given collection.
+
+Decreasing this setting prevents further cursor-aware Eventing functions from being registered on any given collection; however, it doesn't unregister already registered cursor-aware Eventing functions.
+--
+
+[%hardbreaks]
+*Default:* `5`
+*Minimum:* `1`
+*Maximum:* `20`
+{blank}
+a¦ Integer
+
+
 |===
 
 //end::UnivConfig[]

docs/modules/eventing-rest-api/partials/paths/adv_settings_update/http-request.adoc

@@ -128,6 +128,44 @@ curl -XPOST -d '{
 ----
 ====
 
+[#ex-sync-gateway-global]
+.Enable Sync Gateway compatibility for a global function
+====
+This example sets `allow_sync_documents` to `false`, to enable compatibility with Sync Gateway.
+The function is currently paused.
+
+.Curl request
+[source,sh]
+----
+curl -XPOST -d '{
+  "deployment_status": true,
+  "processing_status": false,
+  "allow_sync_documents": false
+}' "http://$ADMIN:$PASSWORD@$HOST:8096/api/v1/functions/my_function/settings"
+----
+
+For details, see xref:sync-gateway::server-compatibility-eventing.adoc[].
+====
+
+[#ex-sync-gateway-scoped]
+.Enable Sync Gateway compatibility for a scoped function
+====
+This example sets `allow_sync_documents` to `false`, to enable compatibility with Sync Gateway.
+The function is currently paused.
+
+.Curl request
+[source,sh]
+----
+curl -XPOST -d '{
+  "deployment_status": true,
+  "processing_status": false,
+  "allow_sync_documents": false
+}' "http://$USER:$PASSWORD@$HOST:8096/api/v1/functions/my_function/settings?bucket=bulk&scope=data"
+----
+
+For details, see xref:sync-gateway::server-compatibility-eventing.adoc[].
+====
+
 .Deploy an undeployed global function -- deprecated
 ====
 .Curl request

docs/modules/eventing-rest-api/partials/paths/config_update/http-request.adoc

@@ -20,6 +20,16 @@ curl -XPOST "http://$ADMIN:$PASSWORD@$HOST:8096/api/v1/config" \
 ----
 ====
 
+.Set cursor limit
+====
+.Curl request
+[source,sh]
+----
+curl -XPOST "http://$ADMIN:$PASSWORD@$HOST:8096/api/v1/config" \
+  -d '{"cursor_limit": 10}'
+----
+====
+
 .Allow interbucket recursion
 ====
 This example disables the safety checks that prevent basic infinite recursive Eventing functions.

src/eventing/eventing.gradle

@@ -1,8 +1,31 @@
+plugins {
+    id("io.datapith.jayOverlay").version("0.1.11")
+}
+
+jayOverlay {
+    targetFile.set("${projectDir}/build/bundle/eventing.yaml")
+    overlayFile.set("${projectDir}/swagger/overlay.json")
+    outputDir.set("${projectDir}/build/overlay")
+}
+
 apply plugin: 'org.openapi.generator'
 
+task bundleSpec(type: org.openapitools.generator.gradle.plugin.tasks.GenerateTask) {
+    generatorName = "openapi-yaml"
+    inputSpec = file("swagger/eventing.yaml").getAbsolutePath().toString()
+    outputDir = "${projectDir}/build/bundle"
+    globalProperties = [
+            generateAliasAsModel: "true"
+    ]
+    additionalProperties = [
+            outputFile: "eventing.yaml",
+    ]
+    generateAliasAsModel = true
+}
+
 openApiGenerate {
     generatorName = "asciidoc"
-    inputSpec = file("swagger/eventing.yaml").getAbsolutePath().toString()
+    inputSpec = file("build/overlay/eventing.yaml").getAbsolutePath().toString()
     outputDir = "${rootDir}/docs/modules/eventing-rest-api/pages"
     templateDir = "${rootDir}/templates"
     gitRepoId = "cb-swagger"
@@ -29,4 +52,7 @@ openApiGenerate {
             depcfg_schema_constants_inner: "Deployment Constants",
             depcfg_schema_curl_inner: "Deployment URL"
     ]
-}
+}
+
+tasks.applyOverlay.dependsOn(bundleSpec)
+tasks.openApiGenerate.dependsOn(applyOverlay)

src/eventing/swagger/eventing.yaml

@@ -615,8 +615,7 @@ paths:
       summary: List Global Config
       description: |-
         Shows all global configuration settings.
-        There are currently just two settings: `enable_debugger` and `ram_quota`.
-        Both of these settings can also be adjusted via the UI.
+        Note that the `enable_debugger` and `ram_quota` settings can also be adjusted via the UI.
       tags:
         - global config
       security:
@@ -841,13 +840,13 @@ components:
 
   schemas:
     settings_schema:
-      $ref: 'https://raw.githubusercontent.com/couchbase/eventing/refs/heads/7.6.3/parser/settings_schema.json'
+      $ref: 'https://raw.githubusercontent.com/couchbase/eventing/refs/heads/master/parser/settings_schema.json'
 
     depcfg_schema:
-      $ref: 'https://raw.githubusercontent.com/couchbase/eventing/refs/heads/7.6.3/parser/depcfg_schema.json'
+      $ref: 'https://raw.githubusercontent.com/couchbase/eventing/refs/heads/master/parser/depcfg_schema.json'
 
     handler_schema:
-      $ref: 'https://raw.githubusercontent.com/couchbase/eventing/refs/heads/7.6.3/parser/handler_schema.json'
+      $ref: 'https://raw.githubusercontent.com/couchbase/eventing/refs/heads/master/parser/handler_schema.json'
 
     AppCode:
       type: string
@@ -893,6 +892,18 @@ components:
           description: |-
             Enables the Eventing service debugger.
             For details, see [Debugging and Diagnosability](/server/8.0/eventing/eventing-debugging-and-diagnosability.html).
+        cursor_limit:
+          type: integer
+          default: 5
+          maximum: 20
+          minimum: 1
+          x-has-default: true
+          description: |-
+            The maximum number of cursor-aware Eventing functions that can coexist on a given source keyspace. (A cursor-aware Eventing function is one for which the `cursor_aware` setting is `true`.)
+
+            Increasing this setting enables more cursor-aware Eventing functions to register and listen to any given collection.
+
+            Decreasing this setting prevents further cursor-aware Eventing functions from being registered on any given collection; however, it doesn't unregister already registered cursor-aware Eventing functions.
 
   responses:
     OK:

src/eventing/swagger/overlay.json

@@ -0,0 +1,27 @@
+{
+  "overlay": "1.0.0",
+  "info": {
+    "title": "Overlay for Eventing Settings Schema",
+    "version": 1.0
+  },
+  "actions": [
+    {
+      "target": "components.schemas.settings_schema.properties.allow_sync_documents",
+      "update": {
+        "description": "Specifies whether the function allows Sync Gateway mutations.\n\n* By default, this setting is `true`, for compatibility with previous versions of Couchbase Server.\n\n* When this setting is `false`, the specified function skips all internal Sync Gateway documents, whose IDs are prefixed with `_sync`.\nThis enables the function to work with Sync Gateway.\n\nYou must ensure that none of the documents which contain your own working data have IDs which are prefixed with `_sync`.\n(Note that internal Sync Gateway attachment documents, whose IDs are prefixed with `_sync:att`, are still processed by the specified function.)"
+      }
+    },
+    {
+      "target": "components.schemas.settings_schema.properties.cursor_aware",
+      "update": {
+        "description": "Specifies whether the function suppresses potential duplicate mutations caused by App Services or Sync Gateway book-keeping.\n\nEnabling this setting guarantees that the Eventing function will only trigger once for any given mutation received from App Services or Sync Gateway.\n\nNote that enabling this setting may have a noticeable impact on the performance of the Eventing function."
+      }
+    },
+    {
+      "target": "components.schemas.settings_schema.properties.cursor_checkpoint_timeout",
+      "update": {
+        "description": "The maximum time the checkpoint writer can run before it is forcefully terminated (in seconds)."
+      }
+    }
+  ]
+}