Nfs example + doc updates for OpenShift newer releases, also fixes issue #45 (#48)

* Adjusting dev100 cpu requirements so pod can be scheduled on a 1CPU node
* Updating helm chart templates
* Fix delete example in documentation 
* Adjust individual names for ClusterRole and ClusterRoleBinding
* Added nfs support
* Added -r switch to restore helm
* Changed dev100 persist to work on Minikube - use default storage class
* Added doc updates for OpenShift newer releases
* Fixed missing timing info in log for config-sync-check at <show><redundancy><detail>
* Updated maintainer details

Author: bczoma

.travis.yml

@@ -37,7 +37,8 @@ script:
   - wget https://raw.githubusercontent.com/$SOLACE_KUBERNETES_QUICKSTART_REPO/$SOLACE_KUBERNETES_QUICKSTART_BRANCH/scripts/configure.sh
   - chmod 755 configure.sh
   - #./configure.sh -c gcp -p admin -i gcr.io/capable-stream-180018/solace-pubsub-standard:8.10.0.1057 -v values-examples/prod1k-direct-noha.yaml
-  - ./configure.sh -c gcp -p admin -i solace/solace-pubsub-standard:8.11.0.1029 -v values-examples/dev100-persist-ha-provisionPvc.yaml
+  - #./configure.sh -c gcp -p admin -i solace/solace-pubsub-standard:8.11.0.1029 -v values-examples/dev100-persist-ha-provisionPvc.yaml
+  - ./configure.sh -c gcp -p admin -v values-examples/dev100-persist-ha-provisionPvc.yaml
   - cd solace-kubernetes-quickstart/solace
   - helm install . -f values.yaml
   - kubectl get statefulset,svc,pods,pvc,pv --show-labels
@@ -54,9 +55,12 @@ script:
   - tar -xvf SDKPERF_C_LINUX64
   - pubSubTools/sdkperf_c -cip=$url -mn=100000 -mr=0 -ptl=t1 -stl=t1 | grep "Total Messages"
   - sleep 30
-  - bash -c 'if [[ -z `curl -sS -u admin:admin http://$url:8080/SEMP -d "<rpc semp-version=\"soltr/8_5VMR\"><show><config-sync></config-sync></show></rpc>" | grep "<oper-status>Up</oper-status>"` ]] ; then echo "config-sync not up!"; exit 1; fi'
-  - curl -sS -u admin:admin http://$url:8080/SEMP -d "<rpc semp-version=\"soltr/8_5VMR\"><show><redundancy></redundancy></show></rpc>"
-  - curl -sS -u admin:admin http://$url:8080/SEMP -d "<rpc semp-version=\"soltr/8_5VMR\"><show><config-sync></config-sync></show></rpc>"
+  - bash -c 'if [[ -z `curl -sS -u admin:admin http://$url:8080/SEMP -d "<rpc><show><config-sync></config-sync></show></rpc>" | grep "<oper-status>Up</oper-status>"` ]] ; then echo "config-sync not up!"; exit 1; fi'
+  - curl -sS -u admin:admin http://$url:8080/SEMP -d "<rpc><show><redundancy></redundancy></show></rpc>"
+  - curl -sS -u admin:admin http://$url:8080/SEMP -d "<rpc><show><config-sync></config-sync></show></rpc>"
+  - helm list
+  - helm delete $(helm list | grep DEPLOYED | awk '{print $1}')
+  - kubectl delete pvc --all
 
 
 after_success:

README.md

@@ -9,7 +9,7 @@ This repository explains how to install a Solace PubSub+ Software Message Broker
 This document is applicable to any platform supporting Kubernetes, with specific hints on how to set up a simple single-node MiniKube deployment on a Unix-based machine. To view examples of other platforms see:
 
 - [Deploying a Solace PubSub+ Software Message Broker HA group onto a Google Kubernetes Engine](https://github.com/SolaceProducts/solace-gke-quickstart )
-- [Deploying a Solace PubSub+ Software Message Broker HA Group onto an OpenShift 3.7 or 3.9 platform](https://github.com/SolaceProducts/solace-openshift-quickstart )
+- [Deploying a Solace PubSub+ Software Message Broker HA Group onto an OpenShift 3.10 or 3.11 platform](https://github.com/SolaceProducts/solace-openshift-quickstart )
 - Deploying a Solace PubSub+ Software Message Broker HA Group onto Amazon EKS (Amazon Elastic Container Service for Kubernetes): follow the [AWS documentation](https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html ) to set up EKS then this guide to deploy.
 
 ## Description of the Solace PubSub+ Software Message Broker
@@ -62,18 +62,18 @@ To load the docker image into a docker registry, follow the steps specific to th
 
 Deploy message broker Pods and Service to the cluster.
 
-The [Kubernetes `helm`](https://github.com/kubernetes/helm/blob/master/README.md ) tool is used to manage this deployment. A deployment is defined by a "helm chart", which consists of templates and values. The values specify the particular configuration properties in the templates.
+The [Kubernetes Helm](https://github.com/kubernetes/helm/blob/master/README.md ) tool is used to manage this deployment. A deployment is defined by a "Helm chart", which consists of templates and values. The values specify the particular configuration properties in the templates.
 
 The following diagram illustrates the template structure used for the Solace Deployment chart. Note that the minimum is shown in this diagram to give you some background regarding the relationships and major functions.
 
 ![alt text](/images/template_relationship.png "Template Relationship")
 
-* First, clone this repo, which includes helper scripts and the `solace` helm chart:
+* First, clone this repo, which includes helper scripts and the `solace` Helm chart:
 
 ```sh
 mkdir ~/workspace; cd ~/workspace
-git clone https://github.com/SolaceDev/solace-kubernetes-quickstart.git
-cd solace-kubernetes-quickstart/solace    # location of the solace helm chart
+git clone https://github.com/SolaceProducts/solace-kubernetes-quickstart.git
+cd solace-kubernetes-quickstart/solace    # location of the solace Helm chart
 ```
 
 * Next, prepare your environment and customize your chart by executing the `configure.sh` script and pass it the required parameters: 
@@ -85,7 +85,7 @@ cd solace-kubernetes-quickstart/solace    # location of the solace helm chart
 | `-c`          | OPTIONAL: The cloud environment you will be running in, current options are [aws\|gcp]. NOTE: if you are not using dynamic provisioned persistent disks, or, if you are running a local MiniKube environment, this option can be left out. |
 | `-v`          | OPTIONAL: The path to a `values.yaml` example/custom file to use. The default file is `values-examples/dev100-direct-noha.yaml` |
 
-The location of the `configure.sh` script is in the `../scripts` directory, relative to the `solace` chart. Executing the configuration script will install the required version of the `helm` tool if needed, as well as customize the `solace` helm chart to your desired configuration.
+The location of the `configure.sh` script is in the `../scripts` directory, relative to the `solace` chart. Executing the configuration script will install the required version of the Helm tool if needed, as well as customize the `solace` Helm chart to your desired configuration.
 
 When customizing the `solace` chart by the script, the `values.yaml` located in the root of the chart will be replaced with what is specified in the argument `-v <value-file>`. A number of examples are provided in the `values-examples/` directory, for details refer to [this section](#other-message-broker-deployment-configurations). 
 
@@ -103,7 +103,7 @@ cd ~/workspace/solace-kubernetes-quickstart/solace
 ../scripts/configure.sh -p <ADMIN_PASSWORD> -i <SOLACE_IMAGE_URL> -c <CLOUD_PROVIDER> -v values-examples/prod1k-persist-ha-provisionPvc.yaml
 ```
 
-* Finally, use `helm` to install the deployment from the `solace` chart location, using your generated `values.yaml` file:
+* Finally, use Helm to install the deployment from the `solace` chart location, using your generated `values.yaml` file:
 
 ```sh
 cd ~/workspace/solace-kubernetes-quickstart/solace
@@ -116,7 +116,7 @@ To modify a deployment, refer to the section [Upgrading/modifying the message br
 
 ### Validate the Deployment
 
-Now you can validate your deployment on the command line. In this example an HA cluster is deployed with po/XXX-XXX-solace-0 being the active message broker/pod. The notation XXX-XXX is used for the unique release name that `helm` dynamically generates, e.g: "tinseled-lamb".
+Now you can validate your deployment on the command line. In this example an HA cluster is deployed with po/XXX-XXX-solace-0 being the active message broker/pod. The notation XXX-XXX is used for the unique release name that Helm dynamically generates, e.g: "tinseled-lamb".
 
 ```sh
 prompt:~$ kubectl get statefulsets,services,pods,pvc,pv
@@ -251,7 +251,20 @@ Use the external Public IP to access the cluster. If a port required for a proto
 
 ## Upgrading/modifying the message broker cluster
 
-To upgrade/modify the message broker cluster, make the required modifications to the chart in the `solace-kubernetes-quickstart/solace` directory as described next, then run the `helm` tool from here. When passing multiple `-f <values-file>` to helm, the override priority will be given to the last (right-most) file specified.
+To upgrade/modify the message broker cluster, make the required modifications to the chart in the `solace-kubernetes-quickstart/solace` directory as described next, then run the Helm tool from here. When passing multiple `-f <values-file>` to Helm, the override priority will be given to the last (right-most) file specified.
+
+### Restoring Helm if not available
+
+Before getting into the details of how to make changes to a deployment, it shall be noted that when using a new machine to access the deployment Helm may not be available. This can be the case when e.g. using cloud shell, which may be terminated any time.
+
+To restore Helm, run the configure command with the -r option:
+
+```
+cd ~/workspace/solace-kubernetes-quickstart/solace
+../scripts/configure.sh -r
+```
+
+Now Helm shall be available, e.g: `helm list` shall no longer return an error message.
 
 ### Upgrading the cluster
 
@@ -344,14 +357,15 @@ Use Helm to delete a deployment, also called a release:
 helm delete XXX-XXX
 ```
 
-> Note: In some versions, Helm may return an error even if the deletion was successful.
-
-Check what has remained from the deployment, which should only return a single line with svc/kubernetes.
+Check what has remained from the deployment, which should only return a single line with svc/kubernetes:
 
 ```
 kubectl get statefulsets,services,pods,pvc,pv
+NAME                           TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)         AGE
+service/kubernetes             ClusterIP      XX.XX.XX.XX     <none>            443/TCP         XX
 ```
-> Note: In some versions, Helm may not be able to clean up all the deployment artifacts, e.g.: pvc/ and pv/. Check their existence with `kubectl get all` and if necessary, use `kubectl delete` to delete those.
+
+> Note: In some versions, Helm may not be able to clean up all the deployment artifacts, e.g.: pvc/ and pv/. If necessary, use `kubectl delete` to delete those.
 
 ## Other Message Broker Deployment Configurations
 
@@ -363,6 +377,7 @@ The `solace-kubernetes-quickstart/solace/values-examples` directory provides exa
 * `prod1k-direct-noha-localDirectory`: production, up to 1000 connections, non-HA, bind the PVC to a local directory on the host node
 * `prod1k-direct-noha-provisionPvc`: production, up to 1000 connections, non-HA, bind the PVC to a provisioned PersistentVolume (PV) in Kubernetes
 * `prod1k-persist-ha-provisionPvc`: production, up to 1000 connections, HA, to bind the PVC to a provisioned PersistentVolume (PV) in Kubernetes
+* `prod1k-persist-ha-nfs`: production, up to 1000 connections, HA, to dynamically bind the PVC to an NFS volume provided by an NFS server, exposed as storage class `nfs`. Note: "root_squash" configuration is supported on the NFS server.
 
 Similar value-files can be defined extending above examples:
 

scripts/configure.sh

@@ -18,7 +18,8 @@
 # The purpose of this script is to:
 #  - take a URL to a Solace PubSub+ docker container, admin password and cloud provider parameters
 #  - install the required version of helm
-#  - clone locally and prepare the solace chart for deployment
+#  - clone locally if not already cloned and prepare the solace chart for deployment
+#  - optionally restore helm installed on both client and server if missing
 
 # Use external env variables if defined: SOLACE_KUBERNETES_QUICKSTART_REPO, SOLACE_KUBERNETES_QUICKSTART_BRANCH
 # otherwise fall back to defaults (defaults are after the - (dash))
@@ -36,16 +37,18 @@ verbose=0
 
 # Read options
 OPTIND=1         # Reset in case getopts has been used previously in the shell.
-while getopts "c:i:p:v:" opt; do
+while getopts "c:i:p:v:r" opt; do
     case "$opt" in
     c)  cloud_provider=$OPTARG   # optional but default will not work in all env
         ;;
     i)  solace_image=$OPTARG     # optional
         ;;
-    p)  solace_password=$OPTARG
+    p)  solace_password=$OPTARG  # optional if using -s (skip values.yaml customization)
         ;;
     v)  values_file=$OPTARG      # optional
         ;;
+    r)  values_file=""           # optional - restore helm only, no values file customization
+        ;;
     esac
 done
 
@@ -79,31 +82,39 @@ case ${os_type} in
     archive_extension="tar.gz"
     sed_options="-E -i.bak"
     sudo_command="sudo"
+    helm_target_location="/usr/bin"
     ;;
   "Linux" )
     helm_type="linux-amd64"
     helm_version="v2.9.1"
     archive_extension="tar.gz"
     sed_options="-i.bak"
     sudo_command="sudo"
+    helm_target_location="/usr/bin"
     ;;
   *_NT* ) # BASH emulation on windows
     helm_type="windows-amd64"
     helm_version=v2.9.1
     archive_extension="zip"
     sed_options="-i.bak"
     sudo_command=""
+    helm_target_location="/usr/bin"
     ;;
 esac
 if exists helm; then
   echo "`date` INFO: Found helm $(helm version --client --short)"
 else
-  pushd /tmp
-  curl -O https://storage.googleapis.com/kubernetes-helm/helm-${helm_version}-${helm_type}.${archive_extension}
-  tar zxf helm-${helm_version}-${helm_type}.${archive_extension} || unzip helm-${helm_version}-${helm_type}.${archive_extension}
-  ${sudo_command} mv ${helm_type}/helm* /usr/bin
-  popd
-  echo "`date` INFO: Installed helm $(helm version --client --short)"
+  if [[ "$helm_type" != "windows-amd64" ]]; then
+    pushd /tmp
+    curl -O https://storage.googleapis.com/kubernetes-helm/helm-${helm_version}-${helm_type}.${archive_extension}
+    tar zxf helm-${helm_version}-${helm_type}.${archive_extension} || unzip helm-${helm_version}-${helm_type}.${archive_extension}
+    ${sudo_command} mv ${helm_type}/helm* $helm_target_location
+    popd
+    echo "`date` INFO: Installed helm $(helm version --client --short)"
+  else
+    echo "Automated install of helm is not supported on Windows. Please refer to https://github.com/helm/helm#install to install it manually then re-run this script."
+    exit  -1
+  fi
 fi
 
 # Deploy tiller
@@ -137,20 +148,25 @@ else
   cd solace
 fi
 
-# Ensure current dir is within the chart - e.g solace-kubernetes-quickstart/solace
-if [ ! -d "templates" ]; then
-  echo "`date` INFO: Must be in the chart directory, exiting. Current dir is $(pwd)."
-  exit -1
+# Copy and customize values.xml
+if [[ "${values_file}" != "" ]]; then
+  # Ensure current dir is within the chart - e.g solace-kubernetes-quickstart/solace
+  if [ ! -d "templates" ]; then
+    echo "`date` INFO: Must be in the chart directory, exiting. Current dir is $(pwd)."
+    exit -1
+  fi
+  echo "`date` INFO: Building helm charts"
+  cp ${values_file} ./values.yaml
+  IFS=':' read -ra container_array <<< "$solace_image"
+  sed ${sed_options} "s:SOLOS_IMAGE_REPO:${container_array[0]}:g" values.yaml
+  tag=${container_array[1]-latest}   # default to latest if no tag provided
+  sed ${sed_options} "s:SOLOS_IMAGE_TAG:${tag}:g"  values.yaml
+  sed ${sed_options} "s/SOLOS_CLOUD_PROVIDER/${cloud_provider}/g"  values.yaml
+  sed ${sed_options} "s/SOLOS_ADMIN_PASSWORD/${solace_password}/g" templates/secret.yaml
+  rm templates/secret.yaml.bak
+else
+  echo "-r option detected, skipping the setup of helm charts."
 fi
-echo "`date` INFO: Building helm charts"
-cp ${values_file} ./values.yaml
-IFS=':' read -ra container_array <<< "$solace_image"
-sed ${sed_options} "s:SOLOS_IMAGE_REPO:${container_array[0]}:g" values.yaml
-tag=${container_array[1]-latest}   # default to latest if no tag provided
-sed ${sed_options} "s:SOLOS_IMAGE_TAG:${tag}:g"  values.yaml
-sed ${sed_options} "s/SOLOS_CLOUD_PROVIDER/${cloud_provider}/g"  values.yaml
-sed ${sed_options} "s/SOLOS_ADMIN_PASSWORD/${solace_password}/g" templates/secret.yaml
-rm templates/secret.yaml.bak
 
 # Wait until helm tiller is up and ready to proceed
 #  workaround until https://github.com/kubernetes/helm/issues/2114 resolved

solace/Chart.yaml

@@ -1,11 +1,11 @@
 apiVersion: v1
 description: Deploy Solace PubSub+ Message Broker Singleton or HA Cluster onto a Kubernetes Cluster
 name: solace
-version: 0.3.0
-icon: https://dev.solace.com/wp-content/uploads/2016/10/solace-logo-white.png
+version: 1.0.1
+icon: https://solace.com/wp-content/uploads/2018/05/Solace_Logo_Green_360x100-1.png
 maintainers:
-  - name: kenbarr
-    email: ken.barr@solace.com
+  - name: support
+    email: support@solace.com
 home: http://dev.solace.com
 sources:
    - https://github.com/SolaceProducts/solace-kubernetes-quickstart

solace/templates/podModRbac.yaml

@@ -1,10 +1,9 @@
 # Fix rbac
----
 {{- if semverCompare ">=1.8-0" .Capabilities.KubeVersion.GitVersion }}
 kind: ClusterRole
 apiVersion: rbac.authorization.k8s.io/v1
 metadata:
-  name: pod-tag-updater
+  name: {{ template "solace.fullname" . }}-podtagupdater
 rules:
 - apiGroups: [""] # "" indicates the core API group
   resources: ["pods"]
@@ -13,11 +12,11 @@ rules:
 kind: ClusterRoleBinding
 apiVersion: rbac.authorization.k8s.io/v1beta1
 metadata:
-  name: solace
+  name: {{ template "solace.fullname" . }}-serviceaccounts-to-podtagupdater
 roleRef:
   apiGroup: rbac.authorization.k8s.io
   kind: ClusterRole
-  name: pod-tag-updater
+  name: {{ template "solace.fullname" . }}-podtagupdater
 subjects:
 - kind: Group
   name: system:serviceaccounts

solace/templates/secret.yaml

@@ -1,4 +1,3 @@
----
 apiVersion: v1
 kind: Secret
 metadata:

solace/templates/service.yaml

@@ -1,5 +1,4 @@
 # Load Service part of template
----
 apiVersion: v1
 kind: Service
 metadata:

solace/templates/solaceConfigMap.yaml

@@ -94,6 +94,7 @@ data:
           role_results=`{{ .Values.filepaths.configmap }}/semp_query.sh -n admin -p ${password} -u http://localhost:8080/SEMP \
                 -q "<rpc><show><redundancy><detail/></redundancy></show></rpc>" \
                 -v "/rpc-reply/rpc/show/redundancy/active-standby-role[text()]"`
+          run_time=$((${count} * ${pause}))
           case "`echo ${role_results} | xmllint -xpath "string(returnInfo/valueSearchResult)" -`" in
             "Primary")
             role="primary"