Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions dist/etc/scylla-manager.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@ http: 127.0.0.1:5080
# local_dc:
#
# Keyspace for management data.
# It will be created automatically on ScyllaDB Manager server startup if it does not already exist.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the manager server created once a cluster is fully constructed? By default RF=3 is used. That requires 3 nodes/racks to already exist. Otherwise, the keyspace will not be created.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See this comment for more context.

# In such case, it will use NetworkTopologyStrategy and provided replication_factor.
# In case different keyspace replication is required, it should be created manually
# before starting ScyllaDB Manager server (this action is recommended for non default
# ScyllaDB Manager database deployments - especially multi datacenter ones).
# keyspace: scylla_manager
# replication_factor: 1
#
Expand Down
43 changes: 21 additions & 22 deletions docs/source/install-scylla-manager.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Install ScyllaDB Manager
:depth: 2
:local:

.. note:: If you need to upgrade Scylla Manager from a previous version please see the :doc:`Upgrade guide <upgrade/index>`.
.. note:: If you need to upgrade ScyllaDB Manager from a previous version please see the :doc:`Upgrade guide <upgrade/index>`.

System requirements
===================
Expand All @@ -18,36 +18,36 @@ While a minimal server can run on a system with 2 cores and 1GB RAM, the followi
* **CPU** - 2vCPUs
* **Memory** - 8GB+ DRAM

.. note:: If you are running `Scylla Monitoring Stack <https://monitoring.docs.scylladb.com/stable/index.html>`_ on the same server as Scylla Manager, your system should also meet the minimal `Monitoring requirements <https://monitoring.docs.scylladb.com/stable/install/monitoring-stack.html#minimal-production-system-recommendations>`_.
.. note:: If you are running `Scylla Monitoring Stack <https://monitoring.docs.scylladb.com/stable/index.html>`_ on the same server as ScyllaDB Manager, your system should also meet the minimal `Monitoring requirements <https://monitoring.docs.scylladb.com/stable/install/monitoring-stack.html#minimal-production-system-recommendations>`_.

Install package
===============

Best practice is to install Scylla Manager Server on a dedicated machine not a Scylla production node.
Download and install the Scylla Manager Server and Client packages from the `Scylla Download Center <https://www.scylladb.com/download/#manager>`_.
Best practice is to install ScyllaDB Manager Server on a dedicated machine not a Scylla production node.
Download and install the ScyllaDB Manager Server and Client packages from the `Scylla Download Center <https://www.scylladb.com/download/#manager>`_.

Configure storage
=================

Scylla Manager uses Scylla to store its data.
You can either use a local one-node Scylla cluster (recommended) or connect Scylla Manager to a remote cluster.
ScyllaDB Manager uses Scylla to store its data.
You can either use a local one-node Scylla cluster (recommended) or connect ScyllaDB Manager to a remote cluster.

Local node
----------

On the same node as you are installing Scylla Manager, download and install Scylla Server package.
On the same node as you are installing ScyllaDB Manager, download and install Scylla Server package.
You can either use `Scylla Enterprise <https://www.scylladb.com/download/#enterprise>`_ or `Scylla Open Source <https://www.scylladb.com/download/#open-source>`_.
There is no need to run the Scylla setup, it is taken care of later, by the ``scyllamgr_setup`` script.
When it's installed you can jump to `Run the scyllamgr_setup script`_ section.

Remote cluster
--------------

Scylla Manager configuration file ``/etc/scylla-manager/scylla-manager.yaml`` contains a database configuration section.
ScyllaDB Manager configuration file ``/etc/scylla-manager/scylla-manager.yaml`` contains a database configuration section.

.. code-block:: yaml

# Scylla Manager database, used to store management data.
# ScyllaDB Manager database, used to store management data.
database:
hosts:
- 127.0.0.1
Expand Down Expand Up @@ -79,13 +79,12 @@ Using an editor open the file and change relevant parameters.

#. If authentication is needed, uncomment and edit the ``user`` and ``password`` parameters.


#. If it's a single DC cluster, uncomment and edit the ``replication_factor`` parameter to match the required replication factor.
This would use SimpleStrategy to create a Scylla Manager keyspace, refer to `Scylla Architecture - Fault Tolerance <https://docs.scylladb.com/architecture/architecture-fault-tolerance/>`_ for more information on replication.
This would use NetworkTopologyStrategy to create a ScyllaDB Manager keyspace, refer to `Scylla Architecture - Fault Tolerance <https://docs.scylladb.com/architecture/architecture-fault-tolerance/>`_ for more information on replication.

#. If it's a multi DC cluster, create a keyspace named ``scylla_manager`` yourself.
You can use a different keyspace name, just remember to adjust the ``keyspace`` parameter value.
Set ``local_dc`` parameter to DC the closest to Scylla Manager Server.
Set ``local_dc`` parameter to DC the closest to ScyllaDB Manager Server.

#. If client/server encryption is enabled, uncomment and set the ``ssl`` parameter to ``true``.
Additional SSL configuration options can be set in the ``ssl`` configuration section.
Expand All @@ -105,7 +104,7 @@ Using an editor open the file and change relevant parameters.
# user_cert_file:
# user_key_file

Sample configuration of Scylla Manager working with a remote cluster with authentication and replication factor 3 could look like this.
Sample configuration of ScyllaDB Manager working with a remote cluster with authentication and replication factor 3 could look like this.

.. code-block:: yaml

Expand All @@ -120,7 +119,7 @@ Sample configuration of Scylla Manager working with a remote cluster with authen
Run the scyllamgr_setup script
==============================

The Scylla Manager setup script automates configuration of Scylla Manager by asking you some questions.
The ScyllaDB Manager setup script automates configuration of ScyllaDB Manager by asking you some questions.
It can be run in non-interactive mode by using flags.

.. code-block:: none
Expand All @@ -130,7 +129,7 @@ It can be run in non-interactive mode by using flags.

Options:
-y, --assume-yes assume that the answer to any question which would be asked is yes
--no-scylla-setup skip setting up and enabling local Scylla instance as a storage backend for Scylla Manager
--no-scylla-setup skip setting up and enabling local Scylla instance as a storage backend for ScyllaDB Manager
--no-enable-service skip enabling service
--no-check-for-updates skip enabling periodic check for updates
-h, --help print this help
Expand All @@ -142,7 +141,7 @@ Run the ``scyllamgr_setup`` script to configure the service.
Enable bash script completion
=============================

Enable bash completion for sctool (the Scylla Manager CLI) in the current bash session.
Enable bash completion for sctool (the ScyllaDB Manager CLI) in the current bash session.
Alternatively, you can just open a new terminal.

.. code-block:: none
Expand All @@ -152,29 +151,29 @@ Alternatively, you can just open a new terminal.
Start ScyllaDB Manager service
===============================

Scylla Manager integrates with ``systemd`` and can be started and stopped using ``systemctl`` command.
ScyllaDB Manager integrates with ``systemd`` and can be started and stopped using ``systemctl`` command.

**Procedure**

#. Start the Scylla Manager server service.
#. Start the ScyllaDB Manager server service.

.. code-block:: none

sudo systemctl start scylla-manager.service

#. Verify the Scylla Manager server service is running.
#. Verify the ScyllaDB Manager server service is running.

.. code-block:: none

sudo systemctl status scylla-manager.service -l
● scylla-manager.service - Scylla Manager Server
● scylla-manager.service - ScyllaDB Manager Server
Loaded: loaded (/usr/lib/systemd/system/scylla-manager.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2019-10-30 11:00:01 UTC; 20s ago
Main PID: 5805 (scylla-manager)
CGroup: /system.slice/scylla-manager.service
└─5805 /usr/bin/scylla-manager

.. note:: The first time you start Scylla Manager it may take a while. It creates a database schema. Follow the logs to see the progress.
.. note:: The first time you start ScyllaDB Manager it may take a while. It creates a database schema. Follow the logs to see the progress.

#. Confirm sctool is running by displaying the sctool version.

Expand All @@ -188,4 +187,4 @@ Scylla Manager integrates with ``systemd`` and can be started and stopped using
Next steps
==========

* :doc:`Install Scylla Manager Agent <install-scylla-manager-agent>`
* :doc:`Install ScyllaDB Manager Agent <install-scylla-manager-agent>`
13 changes: 9 additions & 4 deletions pkg/cmd/scylla-manager/db.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ import (
"text/template"
"time"

"github.com/scylladb/go-log/gocqllog"

"github.com/gocql/gocql"
"github.com/pkg/errors"
"github.com/scylladb/go-log"
"github.com/scylladb/go-log/gocqllog"
"github.com/scylladb/gocqlx/v2"
"github.com/scylladb/gocqlx/v2/dbutil"
"github.com/scylladb/gocqlx/v2/migrate"
Expand Down Expand Up @@ -55,10 +55,15 @@ func createKeyspace(ctx context.Context, c config.Config, logger log.Logger) err
}
}

return session.Query(mustEvaluateCreateKeyspaceStmt(c)).Exec()
ksCreateStmt := mustEvaluateCreateKeyspaceStmt(c)
if err := session.Query(ksCreateStmt).Exec(); err != nil {
return errors.Wrapf(err, "failed to create manager keyspace with %q, "+
"consider creating it manually and starting manager server again", ksCreateStmt)
}
return nil
}

const createKeyspaceStmt = "CREATE KEYSPACE {{.Keyspace}} WITH replication = {'class': 'SimpleStrategy', 'replication_factor': {{.ReplicationFactor}}}"
const createKeyspaceStmt = "CREATE KEYSPACE {{.Keyspace}} WITH replication = {'class': 'NetworkTopologyStrategy', 'replication_factor': {{.ReplicationFactor}}}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know how big this table is in production? When tablets are enabled, it will create 10 tablets/shard by default. Maybe we should start small and let a load balancer split it when needed. \cc @bhalevy

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See this comment for more context.


func mustEvaluateCreateKeyspaceStmt(c config.Config) string {
t := template.New("")
Expand Down
12 changes: 11 additions & 1 deletion pkg/scyllaclient/client_scylla_integration_test.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright (C) 2017 ScyllaDB
// Copyright (C) 2025 ScyllaDB

//go:build all || integration

Expand Down Expand Up @@ -125,10 +125,20 @@ func TestClientDescribeRingIntegration(t *testing.T) {
clusterSession := db.CreateSessionAndDropAllKeyspaces(t, client)
defer clusterSession.Close()

ni, err := client.AnyNodeInfo(t.Context())
if err != nil {
t.Fatal(err)
}

ringDescriber := scyllaclient.NewRingDescriber(context.Background(), client)
for i := range testCases {
tc := testCases[i]
t.Run(tc.name, func(t *testing.T) {
if CheckConstraint(t, ni.ScyllaVersion, ">= 2025.1") && tc.replication != scyllaclient.NetworkTopologyStrategy {
// Tablets need to be manually disabled when using non
// NetworkTopologyStrategy keyspace replication strategy.
tc.replicationStmt += " AND tablets = {'enabled': false}"
}
if err := clusterSession.ExecStmt(fmt.Sprintf("CREATE KEYSPACE %s WITH replication = %s", tc.name, tc.replicationStmt)); err != nil {
t.Fatal(err)
}
Expand Down
4 changes: 2 additions & 2 deletions pkg/service/restore/restore_integration_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ func TestRestoreTablesUserIntegration(t *testing.T) {
Print("Log in via restored user and check permissions")
userSession := CreateManagedClusterSession(t, false, h.dstCluster.Client, user, pass)
newKs := randomizedName("ks_")
ExecStmt(t, userSession, fmt.Sprintf("CREATE KEYSPACE %s WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 2}", newKs))
ExecStmt(t, userSession, fmt.Sprintf("CREATE KEYSPACE %s WITH replication = {'class': 'NetworkTopologyStrategy', 'replication_factor': 2}", newKs))
}

func TestRestoreTablesNoReplicationIntegration(t *testing.T) {
Expand All @@ -74,7 +74,7 @@ func TestRestoreTablesNoReplicationIntegration(t *testing.T) {
ks := randomizedName("no_rep_ks_")
tab := randomizedName("tab_")
Printf("Create non replicated %s.%s in both cluster", ks, tab)
ksStmt := fmt.Sprintf("CREATE KEYSPACE %q WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1}", ks)
ksStmt := fmt.Sprintf("CREATE KEYSPACE %q WITH replication = {'class': 'NetworkTopologyStrategy', 'dc1': 1}", ks)
tabStmt := fmt.Sprintf("CREATE TABLE %q.%q (id int PRIMARY KEY, data int)", ks, tab)
ExecStmt(t, h.srcCluster.rootSession, ksStmt)
ExecStmt(t, h.srcCluster.rootSession, tabStmt)
Expand Down
4 changes: 2 additions & 2 deletions pkg/testutils/db/db.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright (C) 2017 ScyllaDB
// Copyright (C) 2025 ScyllaDB

package db

Expand Down Expand Up @@ -154,7 +154,7 @@ func createTestKeyspace(tb testing.TB, cluster *gocql.ClusterConfig, keyspace st

ExecStmt(tb, session, fmt.Sprintf(`CREATE KEYSPACE %s
WITH replication = {
'class' : 'SimpleStrategy',
'class' : 'NetworkTopologyStrategy',
'replication_factor' : %d
}`, keyspace, 1))
}
Expand Down
Loading