You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow our simple xref:getting-started:index.adoc[getting started guide] to sign up for Astra and get your first streaming tenant created.
7
+
See xref:getting-started:index.adoc[].
8
8
9
-
== How is {product_name} priced?
10
-
{product_name} offers customers a *Pay As You Go*, consumption based pricing model that allows customers to use {product_name} with a cost model that scales as they grow.
9
+
== How is {product} priced?
11
10
12
-
Customers can opt to customize their deployment to meet specific requirements which will adjust their pricing up or down depending on their specific requirement. These customizations options include items such as:
11
+
See xref:operations:astream-pricing.adoc[].
13
12
14
-
* Message retention duration
15
-
* Maximum message retention storage
16
-
* Number of tenants
17
-
* Region of tenant
13
+
== Why is {product} based on Apache Pulsar?
18
14
19
-
== Why did DataStax opt to base {product_name} on Apache Pulsar?
20
-
See our https://www.datastax.com/blog/four-reasons-why-apache-pulsar-essential-modern-data-stack[blog post] that explains why we are excited about Apache Pulsar and why we decided it was the best technology to base {product_name} on.
15
+
For information about the decision to use Apache Pulsar, see https://www.datastax.com/blog/four-reasons-why-apache-pulsar-essential-modern-data-stack[Four Reasons Why Apache Pulsar is Essential to the Modern Data Stack].
21
16
22
-
== What is DataStax plan for Kesque?
23
-
{product_name} is based heavily on technology originally created as part of Kesque. With the launch of {product_name} we will begin the process of shutting down the Kesque service and migrating customers to the new {product_name} platform.
17
+
== What happened to Kesque?
24
18
25
-
== How can I get started with {product_name}?
26
-
To get started with {product_name}, you can create a free account at https://astra.datastax.com and create your first streaming instance immediately. No credit card required.
19
+
{product} is based heavily on technology originally created as part of Kesque.
20
+
With the launch of {product}, {company} began shutting down the Kesque service and migrated customers to {product}.
27
21
28
-
== Who are the target customers for {product_name}?
29
-
{product_name} has been architected to satisfy the most stringent enterprise requirements around availability, scale and latency. {product_name} was built to handle mission critical use cases for Fortune 100 companies across BFSI, Telecommunications, Technology, Retail, Oil & Gas and Healthcare.
22
+
== Who should use {product}?
30
23
31
-
The pricing model also makes {product_name} accessible to mid market and small/medium business customers who need event stream processing capabilities to run core parts of their business.
24
+
{product} has been architected to satisfy the most stringent enterprise requirements around availability, scale, and latency.
25
+
{product} was built to handle mission critical use cases for Fortune 100 companies across BFSI, Telecommunications, Technology, Retail, Oil and Gas, and Healthcare.
32
26
33
-
And finally {product_name} offers a user friendly interface and free tier to satisfy the needs of individual developers and technologists who want to learn more about Apache Pulsar and data streaming in general.
27
+
The pricing model also makes {product} accessible to mid market and small/medium business customers who need event stream processing capabilities to run core parts of their business.
34
28
35
-
== What is CDC for Astra DB?
29
+
And finally {product} offers a user-friendly interface and free tier to satisfy the needs of individual developers and technologists who want to learn more about Apache Pulsar and data streaming in general.
36
30
37
-
Change Data Capture (CDC) for Astra DB uses a Pulsar IO source connector that processes changes from the Cassandra Change Agent via a Pulsar topic. For more, see https://docs.datastax.com/en/astra/docs/astream-cdc.html[CDC for Astra DB].
31
+
== What is CDC for {astra_db}?
32
+
33
+
Change Data Capture (CDC) for {astra_db} uses a Pulsar IO source connector that processes changes from the Cassandra Change Agent via a Pulsar topic.
34
+
For more information, see xref:developing:astream-cdc.adoc[].
38
35
39
36
== What are tenants, namespaces, topics, and sinks?
40
37
41
38
*Tenants* are an isolated administrative unit for which an authorization scheme can be set and a set of clusters can be defined.
42
39
Each tenant can have multiple *namespaces*, a logical container for creating and managing a hierarchy of topics.
43
40
A *topic* is a named channel for transmitting messages from producers to consumers.
44
-
A *sink* feeds data from {product_name} to an external system, such as Cassandra or Elastic Search.
41
+
A *sink* feeds data from {product} to an external system, such as {cassandra} or Elasticsearch.
45
42
46
43
== See also
47
44
48
-
* xref:getting-started:index.adoc[Getting started with Astra Streaming]
Default and xref:astream-custom-roles.adoc[custom roles] allow admins to manage unique permissions for users based on your organization and database requirements.
4
+
You manage role-based access control (RBAC) for {product} through your {astra_db} organization.
5
+
For information about {astra_db} RBAC, including default roles, custom roles, permissions, and user management, see xref:astra-db-serverless:administration:manage-database-access.adoc[].
5
6
6
-
You can manage roles using the {astra_ui} or the https://docs.datastax.com/en/astra/docs/manage/devops/devops-roles.html[DevOps API].
7
+
== {product} permissions
7
8
8
-
== {product_name} Organization permissions
9
+
Permissions specific to {product} include the following:
|Create or modify an access list using the DevOps API or the Astra console.
22
-
|accesslist-write
15
+
There are no default {astra_db} roles specifically scoped to {product}.
16
+
However, the following default roles have the *Manage Streaming* permission:
23
17
24
-
|Delete Custom Role
25
-
|Delete of custom role.
26
-
|org-role-delete
18
+
* *Organization Administrator*
19
+
* *Administrator Service Account*
20
+
* *API Administrator Service Account*
21
+
* *API Administrator User*
27
22
28
-
|Manage Streaming
29
-
|Create a Streaming Service using the DevOps API or the Astra console.
30
-
|org-stream-manage
23
+
For information about permissions assigned to default roles, see xref:astra-db-serverless:administration:manage-database-access.adoc[].
31
24
32
-
|Write Organization
33
-
|Create new organizations or delete an existing organization. Hides manage org and org settings.
34
-
|org-write
25
+
=== Custom roles for {product}
35
26
36
-
|Read Billing
37
-
|Enables links and access to billing details page.
38
-
|org-billing-read
27
+
If you xref:astra-db-serverless:administration:manage-database-access.adoc#custom-roles[create custom roles] for {product}, those roles must have the following permissions, at minimum:
39
28
40
-
|Read IP Access List
41
-
|Enables links and access to access list page.
42
-
|accesslist-read
29
+
* *Manage Streaming* (`org-stream-manage`): View and manage {product} in the {astra_ui}.
30
+
* *View DB* (`org-db-view`): View the {astra_ui} in general.
43
31
44
-
|Read User
45
-
|Access to viewing users of an organization.
46
-
|org-user-read
32
+
Additional permissions might be required, depending on the tasks the role needs to perform.
47
33
48
-
|Read Organization
49
-
|View organization in the Astra console.
50
-
|org-read
34
+
[TIP]
35
+
====
36
+
To control access to specific streaming tenants, you can set granular xref:astra-db-serverless:administration:manage-database-access.adoc#role-scopes[resource scopes] on custom roles.
37
+
====
51
38
52
-
|Read Custom Role
53
-
|See a custom role and its associated permissions.
54
-
|org-role-read
39
+
== Authentication and authorization in Pulsar and {astra_db}
55
40
56
-
|Read External Auth
57
-
|See security settings related to external authentication providers.
58
-
|org-external-auth-read
41
+
Pulsar has the concept of https://pulsar.apache.org/docs/security-authorization/[clients with role tokens].
42
+
In Pulsar, authentication is the process of verifying a provided token (JWT), and authorization is the process of determining if the role claimed in that token is allowed to complete the requested action.
59
43
60
-
|Read Token
61
-
|Read token details.
62
-
|org-token-read
44
+
{product} uses the {company} version of Apache Pulsar (xref:luna-streaming::index.adoc[Luna Streaming]).
45
+
The https://github.com/datastax/pulsar[Luna project] is an open fork of the Pulsar project that maintains feature parity with OSS Pulsar. {product}, as a managed service, abstracts some features/options of Pulsar to ensure continuous, reliable service.
63
46
64
-
|Delete Custom Role
65
-
|Delete of custom role.
66
-
|org-role-delete
47
+
On a shared cluster, your {astra_db} organization has one or more tenants on a shared Pulsar cluster.
48
+
Each of your tenants is secured by Pulsar authentication and authorization models, as well as your {astra_db} organization's authentication and authorization ({astra_db} RBAC).
67
49
68
-
|Notification Write
69
-
|Enable or disable notifications in organization notification settings.
70
-
|org-notification-write
71
-
|Write Billing
72
-
|Enables links and ability to add or edit billing payment info.
73
-
|org-billing-write
50
+
{product} shared clusters are created and administered by {product} administrators.
51
+
Each tenant is assigned a custom role and permissions limited to that tenant only.
52
+
All tokens created within a tenant are assigned roles similar to the assigning tenant.
74
53
75
-
|Write User
76
-
|Add, create, or remove a user using the DevOps API or the Astra console.
77
-
|org-user-write
54
+
For programmatic access, you use {astra_db} application tokens or Pulsar JWT, depending on the operation you need to perform.
55
+
For more information, see xref:operations:astream-token-gen.adoc[].
78
56
79
-
|Write Custom Role
80
-
|Create custom role.
81
-
|org-role-write
57
+
== See also
82
58
83
-
|Write External Auth
84
-
|Update security settings related to external auth providers.
85
-
|org-external-auth-write
86
-
87
-
|Write Token
88
-
|Create application token.
89
-
|org-token-write
90
-
91
-
|===
92
-
93
-
== Astra DB Permissions
94
-
95
-
For documentation about Astra DB user permissions, see https://docs.datastax.com/en/astra/docs/user-permissions.html[Astra DB User Permissions].
*Subscriptions* in Pulsar describe which consumers are consuming data from a topic and how they want to consume that data. +
5
+
_Subscriptions_ in Pulsar describe which consumers are consuming data from a topic and how they want to consume that data.
6
6
7
-
An *exclusive* subscription describes a basic publish-subscribe pattern where a *single consumer* subscribes to a *single topic* and consumes from it. +
7
+
An _exclusive subscription_ describes a basic publish-subscribe (pub-sub) pattern where a single consumer subscribes to a single topic and consumes from it.
8
8
9
-
This document explains how to use Pulsar's exclusive subscription model to manage your topic consumption.
9
+
This page explains how to use Pulsar's exclusive subscription model to manage your topic consumption.
10
10
11
-
include::partial$subscription-prereq.adoc[]
11
+
include::ROOT:partial$subscription-prereq.adoc[]
12
12
13
13
[#example]
14
14
== Exclusive subscription example
15
15
16
-
This example uses the `pulsarConsumer` object in `SimplePulsarConsumer.java` below. +
17
-
16
+
. To configure a Pulsar exclusive subscription, define a `pulsarConsumer` object in `SimplePulsarConsumer.java`, as you would for other subscription types.
17
+
However, you don't need to declare a `subscriptionType`.
18
+
Whereas other subscription types required you to declare a specific `subscriptionType`, Pulsar creates an exclusive subscription by default if you don't declare a `subscriptionType`.
If you want to explicitly define an exclusive subscription, you can add `.subscriptionType(SubscriptionType.Exclusive)` to the consumer.
30
35
31
-
[NOTE]
32
-
====
33
-
Pulsar creates an exclusive subscription by default when no `subscriptionType` is declared.
34
-
====
35
-
36
-
. Open the `pulsar-subscription-example` repo in the IDE of your choice and run `SimplePulsarConsumer.java` to begin consuming messages. +
37
-
The confirmation message and a cursor appear to indicate the consumer is ready.
36
+
. In the `pulsar-subscription-example` project, run `SimplePulsarConsumer.java` to begin consuming messages.
37
+
+
38
+
The confirmation message and a cursor appear to indicate the consumer is ready:
38
39
+
39
-
[source,bash]
40
+
.Result
41
+
[source,console]
40
42
----
41
43
[main] INFO com.datastax.pulsar.Configuration - Configuration has been loaded successfully
42
44
...
43
45
[pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://<tenant_name>/<namespace>/in][SimplePulsarConsumer] Subscribed to topic on <service_url> -- consumer: 0
44
46
----
45
47
46
-
. In a new terminal window, run `SimplePulsarProducer.java` to begin producing messages.
48
+
. In a new terminal window, run `SimplePulsarProducer.java` to begin producing messages:
47
49
+
48
-
[source,bash]
50
+
.Result
51
+
[source,console]
49
52
----
50
53
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 93573631 sent
51
54
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 16931522 sent
52
55
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 68306175 sent
53
56
----
54
-
55
-
. The consumer begins consuming the produced messages.
56
57
+
57
-
[source,bash]
58
+
In the `SimplePulsarConsumer` terminal, the consumer begins consuming the produced messages:
59
+
+
60
+
.Result
61
+
[source,console]
58
62
----
59
63
[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":93573631,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}
60
64
[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":16931522,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}
61
65
[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":68306175,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}
62
66
----
63
67
64
-
. Open a new terminal window and try to run `SimplePulsarConsumer.java`.
68
+
. In a new terminal window, try to run another instance of `SimplePulsarConsumer.java`.
69
+
+
70
+
The second consumer can't subscribe to the topic because the subscription is exclusive:
65
71
+
66
-
[source,bash]
72
+
.Result
73
+
[source,console]
67
74
----
68
75
[main] INFO com.datastax.pulsar.Configuration - Configuration has been loaded successfully
69
76
...
@@ -73,16 +80,9 @@ Exception in thread "main" java.lang.IllegalStateException: Cannot connect to pu
73
80
Caused by: org.apache.pulsar.client.api.PulsarClientException$ConsumerBusyException: {"errorMsg":"Exclusive consumer is already connected","reqId":2964994443801550457, "remote":"<service_url>", "local":"/192.168.0.95:55777"}
74
81
----
75
82
76
-
The second consumer can't subscribe to the topic because the subscription is *exclusive*. +
77
-
78
-
In the example above, the consumer didn't declare a subscription type, so Pulsar created an exclusive subscription by default. +
79
-
To explicitly define an exclusive subscription, add `.subscriptionType(SubscriptionType.Exclusive)` to the consumer.
80
-
81
-
== What's next
82
-
83
-
For more on subscriptions, see:
83
+
== See also
84
84
85
85
* xref:astream-subscriptions.adoc[Subscriptions in Pulsar]
0 commit comments