When considering {product} for your production workloads, you might have questions about what options are available for connecting and consuming your serverless clusters. This page answers some common questions about getting started with and using {product}.
For information about the decision to use {pulsar}, see Four Reasons Why {pulsar} is Essential to the Modern Data Stack.
{product} is based heavily on technology originally created as part of Kesque. With the launch of {product}, {company} began shutting down the Kesque service and migrated customers to {product}.
{product} has been architected to satisfy the most stringent enterprise requirements around availability, scale, and latency. {product} was built to handle mission critical use cases for Fortune 100 companies across BFSI, Telecommunications, Technology, Retail, Oil and Gas, and Healthcare.
The pricing model also makes {product} accessible to mid market and small/medium business customers who need event stream processing capabilities to run core parts of their business.
And finally {product} offers a user-friendly interface and free tier to satisfy the needs of individual developers and technologists who want to learn more about {pulsar} and data streaming in general.
Running a {pulsar-reg} cluster that can handle at-scale workloads is not a trivial task. It requires many (virtual) machines to be configured in a very particular way.
In traditional cloud environments, you would pay hourly for every machine whether they are being used for workloads or not, and you would carry the burden of maintaining the server infrastructure.
A serverless environment removes those operational burdens, and you pay only for the resources you actually use.
Change Data Capture (CDC) for {astra-db} uses a {pulsar-short} IO source connector that processes changes from the {cass-short} Change Agent through a {pulsar-short} topic. For more information, see developing:astream-cdc.adoc.
Tenants are an isolated administrative unit for which an authorization scheme can be set and a set of clusters can be defined. Each tenant can have multiple namespaces, a logical container for creating and managing a hierarchy of topics. A topic is a named channel for transmitting messages from producers to consumers. A sink feeds data from {product} to an external system, such as {cass-reg} or Elasticsearch.
The following questions address differences between shared clusters and Streaming Capacity Units.
See operations:astream-pricing.adoc.
{product} dedicated clusters are now called Streaming Capacity Units.
Connections to both cluster types are secure. However, Streaming Capacity Units offer the option for private connectivity.
For Streaming Capacity Units, all connections in {product} are guarded by authentication, authorization, and secure (TLS) communications. You can connect over the public internet or establish a private connection.
Shared clusters also use a secure connection over the public internet. However, shared clusters don’t support private links.
For more information, see operations:astream-pricing.adoc.
Geo-replication is available for both shared clusters and Streaming Capacity Units. Both cluster types can replicate to other clusters, but there are some differences:
- Connections
-
For shared clusters, traffic between clusters is secured over the public internet.
Streaming Capacity Units can use either the default secure public internet connection or enable private links.
- Regions
-
Shared clusters can replicate between any supported region of the same cloud provider.
Streaming Capacity Units can use almost any region supported by your cloud provider, as well as across cloud providers.
For more on geo-replication, see operations:astream-georeplication.adoc.
The following questions address security and access control in {product}.
If your {product-short} organization is on the Pay As You Go or Enterprise {product-short} subscription plan, your users can use SSO to sign in to the {astra-ui}. For more information, see astra-db-serverless:administration:configure-sso.adoc.
See ROOT:astream-org-permissions.adoc.
The following questions are related to application development with {product}.
Unless you are starting a project from scratch, you likely have message data that needs to be brought over to your {product} tenants. For migration assistance, contact {support_url}[{company} Support].
Every tenant in {product} comes with custom ports for Kafka and RabbitMQ workloads. {company} also offers a fully-compatible JMS implementation for your Java workloads. For more information, see streaming-learning:use-cases-architectures:starlight/index.adoc.
It is common to have a hierarchy of development environments through which you promote app changes before they reach production. The configurations of middleware and platforms supporting the app should be kept in parity to promote stability and fast iterations with low volatility.
- By Tenant
-
To support the hierarchy of development environments, {company} recommends creating separate tenants for each development environment. This gives you the greatest flexibility to balance the separation of roles with consistent service configuration.
All tokens created within a tenant are limited to that tenant.
For example, start with a tenant named
Devthat your development teams can access and create tokens for, and then create other tenants namedStagingandProduction. At each level of the hierarchy, there are fewer users with access to the environment’s tenant, which means fewer opportunities to create tokens that can programmatically access that tenant. Yet, you still maintain parity across the three environments. - By Namespace
-
Alternatively, you might choose to separate development environments by namespace within your {product} tenant. While this doesn’t offer as much flexibility as separation by tenant, it does offer a much simpler model to manage. Also, note that in this scheme you cannot limit access by namespace. All tokens would have access to all namespaces.
{product} is actively maintained to keep parity with the official {pulsar} project. The notable differences arise from accessibility and security. Naturally, you have less control in a managed, serverless cluster than you do in a cluster running in your own environment. Beyond those differences, the effort to develop locally and then move to {product} should not be significant, but {company} recommends that you develop directly in {product}. If you are trying to reduce costs, use the free tier of {product} and then switch when you are ready to stage your production services.
Yes, {product} offers a fully compatible Kafka and RabbitMQ implementation. This means you can use your existing Kafka or RabbitMQ applications with {product}. You can also use the {product} Kafka or RabbitMQ implementation with your existing {pulsar-short} applications. {product} comes with custom ports for Kafka and RabbitMQ workloads. Learn more about the Starlight suite of APIs.
Yes, {product} offers a fully compatible JMS implementation. This means you can use your existing JMS applications with {product}. You can also use the {product} JMS implementation with your existing {pulsar-short} applications. Learn more about the Starlight suite of APIs.