Ruby SR/MR cluster management (#142)

* Python SR/MR cluster management

* Uncomment test

* Minor changes - PR review comments

* Update README.md

* ENV variable consistency

---------

Co-authored-by: Mitchell Elholm <elholmit@amazon.com>

Author: mitchell-elholm

.github/workflows/ruby-cm-integ-tests.yml

@@ -34,10 +34,10 @@ jobs:
     - name: Checkout code
       uses: actions/checkout@v4
 
-    - name: Set up Ruby 2.5
+    - name: Set up Ruby 3.3
       uses: ruby/setup-ruby@v1
       with:
-        ruby-version: '2.5'
+        ruby-version: '3.3'
         bundler-cache: true
 
     - name: Configure AWS Credentials
@@ -48,6 +48,8 @@ jobs:
 
     - name: Configure and run integration for cluster management
       working-directory: ./ruby/cluster_management
+      env:
+        IS_CI: "TRUE"
       run: |
         bundle install
         rspec

ruby/cluster_management/Gemfile

@@ -1,6 +1,4 @@
 source "https://rubygems.org"
 
-gem "aws-sdk-dsql", "~> 1"
-
-gem 'rspec', '3.13.0'
-gem 'aws-sdk-core', '3.211.0'
+gem "aws-sdk-dsql", "~> 1.8"
+gem 'rspec', '3.13.0'

ruby/cluster_management/README.md

@@ -1,30 +1,74 @@
-# Aurora DSQL Ruby SDK code examples
+# Aurora DSQL Ruby code examples
 
 ## Overview
 
-The code examples in this topic show you how to use the AWS Ruby SDK with DSQL to create, read, update, and delete clusters.
+The code examples in this topic show you how to use the AWS Ruby SDK with DSQL
+to create, update, get, and delete single- and multi-Region clusters.
+
+Each file in the [/lib](lib) directory demonstrates a minimal
+working example for each operation. The example function for each operation is invoked
+in [`dsql_cluster_management_spec.rb`](spec/dsql_cluster_management_spec.rb).
 
 ## Run the examples
 
+### ⚠️ Important
+
+* Running this code might result in charges to your AWS account.
+* We recommend that you grant your code least privilege. At most, grant only the
+  minimum permissions required to perform the task. For more information, see
+  [Grant least privilege](https://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html#grant-least-privilege).
+* This code is not tested in every AWS Region. For more information, see
+  [AWS Regional Services](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services).
+
 ### Prerequisites
 
-* Ruby version >=3.3 is needed
+- Ruby version >= 3.3 is installed.
+  - **MacOS Optional:** Use rbenv to manage Ruby version
 
-### Setup test running environment 
+```bash
+# Optional use rbenv
+rbenv install 3.3.5
+rbenv local 3.3.5
 
-Ensure you are authenticated with AWS credentials. No other setup is needed besides having Ruby installed.
+ruby --version
+```
+
+- Valid AWS credentials can be discovered by
+  the [default provider chain](https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/credential-providers.html).
 
-### Run the example tests
+### Execute tests to create and delete clusters
 
-In a terminal run the following command from the crud_example folder:
 ```sh
+# Optional: Single-Region examples will execute in CLUSTER_REGION. Defaults to 'us-east-1'.
+export CLUSTER_REGION="us-east-1"
+
+# Optional: Multi-Region examples will create clusters in CLUSTER_1_REGION and CLUSTER_2_REGION
+# with WITNESS_REGION as witness for both. Defaults to 'us-east-1' for CLUSTER_1_REGION, 'us-east-2' 
+# for CLUSTER_2_REGION and 'us-west-2' for WITNESS_REGION.
+export CLUSTER_1_REGION="us-east-1"
+export CLUSTER_2_REGION="us-east-2"
+export WITNESS_REGION="us-west-2"
+
 bundle install
 
+# Will create, update, read, then delete clusters
 rspec
 ```
 
+Test execution will take around five minutes as it waits for clusters to complete activation and deletion.
+
+### Executing single operations
+
+Files in [lib/](lib/) each have a `main()` function that let you exercise single operations.
+
+```shell
+# Check each operation for its expected environment variables
+CLUSTER_REGION="us-east-1" CLUSTER_ID="<your cluster id>" \
+  ruby lib/create_single_region_cluster.rb
+```
+
 ---
 
-Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. 
+Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
 
-SPDX-License-Identifier: MIT-0
+SPDX-License-Identifier: MIT-0

ruby/cluster_management/lib/create_multi_region.rb

@@ -0,0 +1,79 @@
+require "aws-sdk-dsql"
+require "pp"
+
+def create_multi_region_clusters(region_1, region_2, witness_region)
+  client_1 = Aws::DSQL::Client.new(region: region_1)
+  client_2 = Aws::DSQL::Client.new(region: region_2)
+
+  # We can only set the witness region for the first cluster
+  puts "Creating cluster in #{region_1}"
+  cluster_1 = client_1.create_cluster(
+    deletion_protection_enabled: true,
+    multi_region_properties: {
+      witness_region: witness_region
+    },
+    tags: {
+      Name: "Ruby-CM-Example-Multi-Region",
+      Repo: "aws-samples/aurora-dsql-samples"
+    }
+  )
+  puts "Created #{cluster_1.arn}"
+
+  # For the second cluster we can set witness region and designate cluster_1 as a peer
+  puts "Creating cluster in #{region_2}"
+  cluster_2 = client_2.create_cluster(
+    deletion_protection_enabled: true,
+    multi_region_properties: {
+      witness_region: witness_region,
+      clusters: [ cluster_1.arn ]
+    },
+    tags: {
+      Name: "Ruby-CM-Example-Multi-Region",
+      Repo: "aws-samples/aurora-dsql-samples"
+    }
+  )
+  puts "Created #{cluster_2.arn}"
+
+  # Now that we know the cluster_2 arn we can set it as a peer of cluster_1
+  client_1.update_cluster(
+    identifier: cluster_1.identifier,
+    multi_region_properties: {
+      witness_region: witness_region,
+      clusters: [ cluster_2.arn ]
+    }
+  )
+  puts "Added #{cluster_2.arn} as a peer of #{cluster_1.arn}"
+
+  # Now that multi_region_properties is fully defined for both clusters
+  # they'll begin the transition to ACTIVE
+  puts "Waiting for #{cluster_1.arn} to become ACTIVE"
+  cluster_1 = client_1.wait_until(:cluster_active, identifier: cluster_1.identifier) do |w|
+    # Wait for 5 minutes
+    w.max_attempts = 30
+    w.delay = 10
+  end
+
+  puts "Waiting for #{cluster_2.arn} to become ACTIVE"
+  cluster_2 = client_2.wait_until(:cluster_active, identifier: cluster_2.identifier) do |w|
+    w.max_attempts = 30
+    w.delay = 10
+  end
+
+  [ cluster_1, cluster_2 ]
+rescue Aws::Errors::ServiceError => e
+  abort "Failed to create multi-region clusters: #{e.message}"
+end
+
+def main
+  region_1 = ENV.fetch("CLUSTER_1_REGION", "us-east-1")
+  region_2 = ENV.fetch("CLUSTER_2_REGION", "us-east-2")
+  witness_region = ENV.fetch("WITNESS_REGION", "us-west-2")
+
+  cluster_1, cluster_2 = create_multi_region_clusters(region_1, region_2, witness_region)
+
+  puts "Created multi region clusters:"
+  pp cluster_1
+  pp cluster_2
+end
+
+main if $PROGRAM_NAME == __FILE__

ruby/cluster_management/lib/create_multi_region_clusters.rb

@@ -0,0 +1,33 @@
+require "aws-sdk-dsql"
+require "pp"
+
+def create_cluster(region)
+  client = Aws::DSQL::Client.new(region: region)
+  cluster = client.create_cluster(
+    deletion_protection_enabled: true,
+    tags: {
+      Name: "Ruby-CM-Example-Single-Region",
+      Repo: "aws-samples/aurora-dsql-samples"
+    }
+  )
+  puts "Created #{cluster.arn}"
+
+  # The DSQL SDK offers built-in waiters to poll for a cluster's
+  # transition to ACTIVE.
+  puts "Waiting for cluster to become ACTIVE"
+  client.wait_until(:cluster_active, identifier: cluster.identifier) do |w|
+    # Wait for 5 minutes
+    w.max_attempts = 30
+    w.delay = 10
+  end
+rescue Aws::Errors::ServiceError => e
+  abort "Unable to create cluster in #{region}: #{e.message}"
+end
+
+def main
+  region = ENV.fetch("CLUSTER_1_REGION", "us-east-1")
+  cluster = create_cluster(region)
+  pp cluster
+end
+
+main if $PROGRAM_NAME == __FILE__

ruby/cluster_management/lib/create_single_region.rb

@@ -0,0 +1,43 @@
+require "aws-sdk-dsql"
+
+def delete_multi_region_clusters(region_1, cluster_id_1, region_2, cluster_id_2)
+  client_1 = Aws::DSQL::Client.new(region: region_1)
+  client_2 = Aws::DSQL::Client.new(region: region_2)
+
+  puts "Deleting cluster #{cluster_id_1} in #{region_1}"
+  client_1.delete_cluster(identifier: cluster_id_1)
+
+  # cluster_1 will stay in PENDING_DELETE state until cluster_2 is deleted
+  puts "Deleting #{cluster_id_2} in #{region_2}"
+  client_2.delete_cluster(identifier: cluster_id_2)
+
+  # Now that both clusters have been marked for deletion they will transition
+  # to DELETING state and finalize deletion
+  puts "Waiting for #{cluster_id_1} to finish deletion"
+  client_1.wait_until(:cluster_not_exists, identifier: cluster_id_1) do |w|
+    # Wait for 5 minutes
+    w.max_attempts = 30
+    w.delay = 10
+  end
+
+  puts "Waiting for #{cluster_id_2} to finish deletion"
+  client_2.wait_until(:cluster_not_exists, identifier: cluster_id_2) do |w|
+    # Wait for 5 minutes
+    w.max_attempts = 30
+    w.delay = 10
+  end
+rescue Aws::Errors::ServiceError => e
+  abort "Failed to delete multi-region clusters: #{e.message}"
+end
+
+def main
+  region_1 = ENV.fetch("CLUSTER_1_REGION", "us-east-1")
+  cluster_id_1 = ENV.fetch("CLUSTER_1_ID")
+  region_2 = ENV.fetch("CLUSTER_2_REGION", "us-east-2")
+  cluster_id_2 = ENV.fetch("CLUSTER_2_ID")
+
+  delete_multi_region_clusters(region_1, cluster_id_1, region_2, cluster_id_2)
+  puts "Deleted #{cluster_id_1} in #{region_1} and #{cluster_id_2} in #{region_2}"
+end
+
+main if $PROGRAM_NAME == __FILE__