You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
. Select the namespace and one or more input topics for the function.
2
+
3
+
. If needed, select an output namespace and topics.
4
+
+
5
+
Whether you need to specify an output topic depends on the function's logic.
6
+
For example, the `cast` xref:developing:astream-functions-transform.adoc[transform function] needs an output topic to receive the transformed messages.
7
+
8
+
. Optional: Select a log topic.
9
+
+
10
+
The log topic is a separate output topic for messages containing additional `loglevel`, `fqn`, and `instance` properties.
11
+
12
+
. Optional: Configure advanced settings:
13
+
+
14
+
* *Instances*: Enter a number of sink instances to run.
15
+
* *Processing Guarantee*: Select one of the following:
16
+
***ATLEAST_ONCE* (default): Each message sent to the function can be processed one or more times.
17
+
***ATMOST_ONCE*: Each message sent to the function is processed 0 or 1 times.
18
+
This means there is a change that a message is not processed.
19
+
***EFFECTIVELY_ONCE*: Each message sent to the function has only one output associated with it.
20
+
* *Timeout*: Set a timeout limit.
21
+
* *Auto Acknowledge*: Enable or disable automatic message acknowledgment.
22
+
23
+
. If needed, provide a JSON-formatted config key for your function.
24
+
This configuration depends on your function code.
25
+
26
+
. Click *Create*.
27
+
28
+
. Confirm your function was created on the *Functions* tab.
29
+
+
30
+
You can also verify the function deployment with the {pulsar-short} CLI:
31
+
+
32
+
[source,shell]
33
+
----
34
+
./bin/pulsar-admin functions list --tenant $TENANT
35
+
----
36
+
37
+
Once created, the function starts processing messages from the input topics according to its logic.
Copy file name to clipboardExpand all lines: modules/developing/pages/astream-cdc.adoc
+25-52Lines changed: 25 additions & 52 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -269,11 +269,7 @@ After you <<create-tenant,create a tenant>> and <<create-table,create tables>>,
269
269
270
270
For <<multi-region-cdc,multi-region databases>>, you must use the xref:astra-db-serverless:api-reference:devops-api.adoc[{product-short} {devops-api}] to enable CDC in secondary regions.
271
271
272
-
[tabs]
273
-
======
274
-
{astra-ui}::
275
-
+
276
-
--
272
+
==== Enable CDC in the {astra-ui}::
277
273
. In the {astra-ui}, click the name of the database where you want to enable CDC.
278
274
+
279
275
If you created the demo table in <<create-table>>, select the database where you created that table.
@@ -291,11 +287,9 @@ If you created the demo table in <<create-table>>, select the database where you
291
287
Enabling CDC on any table disables the *Add a region* functionality in the {astra-ui} for that database.
292
288
You must use the {product-short} {devops-api} to add a region after enabling CDC.
293
289
====
294
-
--
295
290
296
-
{product-short} {devops-api}::
297
-
+
298
-
--
291
+
==== Enable CDC with the {product-short} {devops-api}
292
+
299
293
[IMPORTANT]
300
294
====
301
295
CDC for multi-region {astra-db} {db-serverless-vector} databases is available only to qualified participants in this private preview release.
@@ -307,10 +301,9 @@ If you're interested in this private preview feature, contact your {company} acc
307
301
308
302
Use the {product-short} {devops-api} to enable CDC on one or more tables in the same database in the same request.
309
303
310
-
You can use these endpoints to enable CDC in single-region and multi-region databases.
304
+
You can use the following endpoints to enable CDC in single-region and multi-region databases.
305
+
To check the CDC configuration for a database or table, see <<get-cdc>>.
311
306
312
-
[tabs]
313
-
====
314
307
Enable CDC after deploying a region::
315
308
+
316
309
Use these steps to enable CDC in a single-region database or in previously-deployed regions of a multi-region database.
@@ -431,11 +424,8 @@ The tenant must be in the same region as specified in `region`.
431
424
+
432
425
All tables replicated to the new region automatically have CDC enabled.
433
426
The `astracdc` namespace and CDC topics for each table are created in the specified regional tenant.
434
-
====
435
427
436
-
To check the CDC configuration for a database or table, see <<get-cdc>>.
437
-
--
438
-
======
428
+
==== CDC topics
439
429
440
430
When you enable CDC on a database for the first time, {astra-db} automatically creates an `astracdc` namespace in your streaming tenant.
441
431
For each table where you enable CDC, {astra-db} creates two topics in the `astracdc` namespace:
@@ -789,23 +779,16 @@ However, be aware of the following limitations:
789
779
790
780
You can check your active CDC configurations in the {astra-ui} or with the {product-short} {devops-api}.
791
781
792
-
[tabs]
793
-
======
794
-
{astra-ui}::
795
-
+
796
-
--
782
+
=== Check CDC status in the {astra-ui}
783
+
797
784
. In the {astra-ui}, click the name of the database that you want to inspect.
798
785
799
786
. Click the *CDC* tab, and then review the list of tables where you have enabled CDC.
800
787
801
788
. Click a table's name to inspect the table's CDC configuration.
802
-
--
803
789
804
-
{product-short} {devops-api}::
805
-
+
806
-
--
807
-
[tabs]
808
-
====
790
+
=== Check CDC status with the {product-short} {devops-api}
791
+
809
792
By database::
810
793
+
811
794
Use `{devops-api-ref-url}#tag/Database-Operations/operation/getCDC[GET /v3/databases/**DB_ID**/cdc]` to get CDC configuration details for an entire database:
@@ -889,9 +872,6 @@ For multi-region databases with CDC enabled in multiple regions, the `regions` a
889
872
}
890
873
]
891
874
----
892
-
====
893
-
--
894
-
======
895
875
896
876
[#put-cdc]
897
877
=== Update a database's CDC configuration
@@ -1036,23 +1016,30 @@ CDC is automatically disabled if you drop a table, terminate a database, or remo
1036
1016
You can remove a table's CDC configuration without deleting the table.
1037
1017
For multi-region databases, this disables CDC for the table across all regions.
1038
1018
1039
-
[tabs]
1040
-
======
1041
-
{astra-ui}::
1019
+
Disabling CDC doesn't remove the associated namespace, topics, or {astra-stream} tenant:
1020
+
1021
+
* If you reenable CDC for the same table, the existing topics are reused with the existing records.
1022
+
1023
+
* If you want to discard a table's CDC records, you must manually remove the associated {astra-stream} artifacts after disabling CDC:
1042
1024
+
1043
-
--
1025
+
** If you remove CDC from a table, then you can delete the table's `data-` and `log-` topics from the `astracdc` namespace in the {astra-stream} tenant.
1026
+
For multi-region databases, make sure that you delete the topics in the tenant for each region where the database is deployed
1027
+
** If you remove CDC from all tables in a region, and you no longer need CDC in that region, then you can delete the `astracdc` namespace from the {astra-stream} tenant in that region.
1028
+
Deleting the namespace also deletes the topics within that namespace.
1029
+
** If you remove CDC from all of a database's tables, and you no longer need an {astra-stream} tenant for any reason, you can delete the entire tenant.
1030
+
1031
+
==== Disable CDC with the {astra-ui}
1032
+
1044
1033
. In the {astra-ui}, click the name of the database where you want to disable CDC.
1045
1034
1046
1035
. Click the *CDC* tab.
1047
1036
1048
1037
. In the *Change Data Capture* list, click the table's name.
1049
1038
1050
1039
. Click *Disable* to remove the table's CDC configuration.
1051
-
--
1052
1040
1053
-
{product-short} {devops-api}::
1054
-
+
1055
-
--
1041
+
==== Disable CDC with the {product-short} {devops-api}::
1042
+
1056
1043
Use `{devops-api-ref-url}#tag/Database-Operations/operation/deleteCDC[DELETE /v3/databases/**DB_ID**/cdc]` to disable CDC for a table:
1057
1044
1058
1045
[source,curl,subs="+quotes"]
@@ -1083,20 +1070,6 @@ If you want to disable CDC for multiple tables at once, include an object for ea
1083
1070
You can use `PUT /v3/databases/**DB_ID**/cdc` to make multiple changes to a database's CDC configuration in one request, including additions, changes, and removals of tables and regions.
1084
1071
For more information, see <<put-cdc>>.
1085
1072
====
1086
-
--
1087
-
======
1088
-
1089
-
Disabling CDC doesn't remove the associated namespace, topics, or {astra-stream} tenant:
1090
-
1091
-
* If you reenable CDC for the same table, the existing topics are reused with the existing records.
1092
-
1093
-
* If you want to discard a table's CDC records, you must manually remove the associated {astra-stream} artifacts after disabling CDC:
1094
-
+
1095
-
** If you remove CDC from a table, then you can delete the table's `data-` and `log-` topics from the `astracdc` namespace in the {astra-stream} tenant.
1096
-
For multi-region databases, make sure that you delete the topics in the tenant for each region where the database is deployed
1097
-
** If you remove CDC from all tables in a region, and you no longer need CDC in that region, then you can delete the `astracdc` namespace from the {astra-stream} tenant in that region.
1098
-
Deleting the namespace also deletes the topics within that namespace.
1099
-
** If you remove CDC from all of a database's tables, and you no longer need an {astra-stream} tenant for any reason, you can delete the entire tenant.
0 commit comments