Skip to content

Commit 46e61f3

Browse files
balaharish7survivant
balaharish7
authored and
survivant
committed
docs: created a new documentation for openshift (openebs#502)
* docs: created a new documentation for openshift Signed-off-by: Bala Harish <[email protected]> * docs: modified the docs and created a new openshift doc for 4.1 Signed-off-by: Bala Harish <[email protected]> --------- Signed-off-by: Bala Harish <[email protected]> Signed-off-by: Sebastien Dionne <[email protected]>
1 parent b346213 commit 46e61f3

File tree

4 files changed

+554
-8
lines changed

4 files changed

+554
-8
lines changed

‎docs/main/Solutioning/openebs-on-kubernetes-platforms/openshift.md‎

Lines changed: 268 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,268 @@
1+
---
2+
id: openshift
3+
title: Replicated PV Mayastor Installation on OpenShift
4+
keywords:
5+
- Replicated PV Mayastor Installation on OpenShift
6+
- Replicated PV Mayastor - Platform Support
7+
- Platform Support
8+
- OpenShift
9+
description: This section explains about the Platform Support for Replicated PV Mayastor.
10+
---
11+
# Replicated PV Mayastor Installation on OpenShift
12+
13+
This document provides instructions for installing Replicated PV Mayastor on OpenShift. Using OpenEBS Replicated PV Mayastor with OpenShift offers several benefits for persistent storage management in Kubernetes environments, especially in the context of DevOps and Cloud-Native applications.
14+
15+
**Cloud-Native and Container-Aware Storage:** OpenEBS is designed to work in a cloud-native, containerized environment that aligns well with OpenShift's architecture. It offers Container Native Storage (CNS), which runs as microservices in the Kubernetes cluster, providing dynamic storage provisioning with high flexibility.
16+
17+
**Dynamic and Scalable Storage:** OpenEBS allows the provisioning of persistent volumes dynamically. This is particularly useful in OpenShift environments where applications may scale rapidly, and on demand, with minimal manual intervention.
18+
19+
**Storage for Stateful Applications:** OpenShift often hosts stateful applications like databases (MySQL, PostgreSQL, Cassandra), message queues, and other services requiring persistent storage. OpenEBS supports various storage engines, such as Replicated PV Mayastor enabling optimized storage performance depending on the workload type.
20+
21+
**Simplified Storage Operations:** With OpenEBS, storage can be managed and operated by DevOps teams without requiring specialized storage administrators. It abstracts away the complexity of traditional storage solutions, providing a Kubernetes-native experience.
22+
23+
**Easy Integration with OpenShift Features:** OpenEBS can integrate seamlessly with OpenShift’s features like Operators, pipelines, and monitoring tools, making it easier to manage and monitor persistent storage using OpenShift-native tools.
24+
25+
## Prerequisites
26+
27+
Before installing Replicated PV Mayastor, make sure that you meet the following requirements:
28+
29+
- **Hardware Requirements**
30+
31+
Your machine type must meet the requirements defined in the [prerequisites](../../user-guides/replicated-storage-user-guide/replicated-pv-mayastor/rs-installation.md#prerequisites).
32+
33+
- **Worker Nodes**
34+
35+
The number of worker nodes on which IO engine pods are deployed should not be less than the desired replication factor when using the synchronous replication feature (N-way mirroring).
36+
37+
- **Additional Disks**
38+
39+
Your worker nodes should have additional storage disks attached. The additional storage disks should not be mounted or contain a filesystem.
40+
41+
- **Enable Huge Pages**
42+
43+
2MiB-sized Huge Pages must be supported and enabled on the storage nodes i.e. nodes where IO engine pods are deployed. A minimum number of 1024 such pages (i.e. 2GiB total) must be available exclusively to the IO engine pod on each node.
44+
Huge pages in the OpenShift Container Platform (OCP) can be enabled during the installation or it can be enabled by creating new machine configs post-installation. Refer to the [Red Hat Documentation](https://access.redhat.com/solutions/5214791) for more details.
45+
46+
- **Kernel Modules**
47+
48+
nvme modules are loaded by default in coreOS.
49+
50+
- **Preparing the Cluster**
51+
52+
Refer to the [Replicated PV Mayastor Installation Documentation](../../user-guides/replicated-storage-user-guide/replicated-pv-mayastor/rs-installation.md#preparing-the-cluster) for instructions on preparing the cluster.
53+
54+
- **Security Context Constraint (SCC)**
55+
56+
Ensure that the service account used for the OpenEBS deployments is added to the privileged SCC.
57+
58+
```
59+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-promtail
60+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-loki
61+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-localpv-provisioner
62+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-nats
63+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-lvm-controller-sa
64+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-lvm-node-sa
65+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-promtail
66+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-service-account
67+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-zfs-controller-sa
68+
oc adm policy -n openebs add-scc-to-user privileged -z openebs-zfs-node-sa
69+
oc adm policy -n openebs add-scc-to-user privileged -z default
70+
```
71+
72+
## Install Replicated PV Mayastor on OpenShift
73+
74+
Refer to the [OpenEBS Installation Documentation](../../quickstart-guide/installation.md#installation-via-helm) to install Replicated PV Mayastor using Helm.
75+
76+
- **Helm Install Command**
77+
78+
```
79+
helm install openebs --namespace openebs openebs/openebs --create-namespace --set openebs-crds.csi.volumeSnapshots.enabled=false
80+
```
81+
82+
:::info
83+
OCP includes VolumeSnapshot CRDs by default. To avoid potential installation issues, it is recommended to disable these CRDs in the OpenEBS Helm chart, as these resources already exist in the OCP environment.
84+
:::
85+
86+
## Pools
87+
88+
The available worker nodes can be viewed using the `kubectl-mayastor` plugin. To use this functionality, you must install `kubectl` (or execute the binary using `./kubectl-mayastor`). The plugin is not compatible with the `oc` binary directly.
89+
90+
```
91+
kubectl mayastor get block-devices NODE_ID -n openebs
92+
```
93+
94+
It is highly recommended to specify the disk using a unique device link that remains unaltered across node reboots. Examples of such device links are: by-path or by-id (Sample disk-pools as below):
95+
96+
**Command**
97+
98+
```
99+
kubectl mayastor get nodes -n openebs
100+
```
101+
102+
**Output**
103+
104+
```
105+
ID GRPC ENDPOINT STATUS VERSION
106+
worker 10.200.31.4:10124 Online v2.7.1
107+
```
108+
109+
**Command**
110+
111+
```
112+
kubectl mayastor get block-devices worker -n openebs
113+
```
114+
115+
**Output**
116+
117+
```
118+
DEVNAME DEVTYPE SIZE AVAILABLE MODEL DEVPATH MAJOR MINOR DEVLINKS
119+
/dev/sdb disk 30GiB yes Virtual_disk /devices/pci0000:00/0000:00:10.0/host2/target2:0:1/2:0:1:0/block/sdb 8 16 "/dev/disk/by-id/scsi-SVMware_Virtual_disk_6000c2915164f6cc7af0aa6cb040cf67", "/dev/disk/by-id/wwn-0x6000c2915164f6cc7af0aa6cb040cf67", "/dev/disk/by-id/scsi-36000c2915164f6cc7af0aa6cb040cf67", "/dev/disk/by-diskseq/2", "/dev/disk/by-path/pci-0000:00:10.0-scsi-0:0:1:0"
120+
```
121+
122+
The status of DiskPools can be determined by referencing their corresponding cluster Custom Resources (CRs). Pools that are available and healthy should report their state as `online`. Verify that the expected number of pools has been created and that all are in the "online" state.
123+
124+
**Command**
125+
126+
```
127+
oc get dsp -n openebs
128+
```
129+
130+
**Output**
131+
132+
```
133+
NAME NODE STATE POOL_STATUS CAPACITY USED AVAILABLE
134+
pool-on-worker worker Created Online 32178700288 0 32178700288
135+
```
136+
137+
## Configuration
138+
139+
- Refer to the [Replicated PV Mayastor Configuration Documentation](../../user-guides/replicated-storage-user-guide/replicated-pv-mayastor/rs-configuration.md#create-replicated-pv-mayastor-storageclasss) for instructions regarding StorageClass creation.
140+
141+
Replicated PV Mayastor dynamically provisions Persistent Volumes (PVs) based on StorageClass definitions that you create. Parameters of the definition are used to set the characteristics and behaviour of its associated PVs. Most importantly StorageClass definition is used to control the level of data protection afforded to it (i.e. the number of synchronous data replicas that are maintained for purposes of redundancy). It is possible to create any number of StorageClass definitions, spanning all permitted parameter permutations. An example is given below:
142+
143+
```
144+
cat <<EOF | oc create -f -
145+
apiVersion: storage.k8s.io/v1
146+
kind: StorageClass
147+
metadata:
148+
name: mayastor-3
149+
parameters:
150+
protocol: nvmf
151+
repl: "3"
152+
provisioner: io.openebs.csi-mayastor
153+
EOF
154+
storageclass.storage.k8s.io/mayastor-3 created
155+
```
156+
157+
- Refer to the [Deploy an Application Documentation](../../user-guides/replicated-storage-user-guide/replicated-pv-mayastor/rs-deployment.md) for instructions regarding PVC creation and deploying an application.
158+
159+
If all verification steps in the preceding stages were satisfied, then the Replicated PV Mayastor has been successfully deployed within the cluster. To verify basic functionality, we will now dynamically provision a Persistent Volume based on a Replicated PV Mayastor StorageClass, mount that volume within a small test pod which we'll create, and use the Flexible I/O Tester utility within that pod to check that I/O to the volume is processed correctly.
160+
161+
Use `oc` to create a PVC based on a StorageClass created. In the example shown below, we will consider StorageClass to have been named "openebs-single-replica" which was created as part of OpenEBS Installation.
162+
163+
```
164+
cat <<EOF | kubectl create -f -
165+
apiVersion: v1
166+
kind: PersistentVolumeClaim
167+
metadata:
168+
name: ms-volume-claim
169+
spec:
170+
accessModes:
171+
- ReadWriteOnce
172+
resources:
173+
requests:
174+
storage: 1Gi
175+
storageClassName: openebs-single-replica
176+
EOF
177+
persistentvolumeclaim/ms-volume-claim created
178+
```
179+
180+
As a next step verify the PV/PVC and the Replicated PV Mayastor volumes.
181+
182+
**Command**
183+
184+
```
185+
oc get pvc
186+
```
187+
188+
**Output**
189+
190+
```
191+
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
192+
ms-volume-claim Bound pvc-144d54db-a3cf-4194-821d-34eae9dafc1d 1Gi RWO openebs-single-replica <unset> 40s
193+
```
194+
195+
**Command**
196+
197+
```
198+
oc get pv
199+
```
200+
201+
**Output**
202+
203+
```
204+
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE
205+
pvc-02333bf8-8a07-4ce0-a00e-bd6bc67af380 2Gi RWO Delete Bound openebs/data-openebs-etcd-0 mayastor-etcd-localpv <unset> 47h
206+
pvc-144d54db-a3cf-4194-821d-34eae9dafc1d 1Gi RWO Delete Bound default/ms-volume-claim openebs-single-replica <unset> 42s
207+
pvc-233aafb1-59e9-4836-b8a1-f74ab2f5a6e4 10Gi RWO Delete Bound openebs/storage-openebs-loki-0 mayastor-loki-localpv <unset> 47h
208+
```
209+
210+
**Command**
211+
212+
```
213+
kubectl mayastor get volumes -n openebs
214+
```
215+
216+
**Output**
217+
218+
```
219+
ID REPLICAS TARGET-NODE ACCESSIBILITY STATUS SIZE THIN-PROVISIONED ALLOCATED SNAPSHOTS SOURCE
220+
144d54db-a3cf-4194-821d-34eae9dafc1d 1 <none> <none> Online 1GiB false 1GiB 0 <none>
221+
```
222+
223+
The Replicated PV Mayastor CSI driver will cause the application pod and the corresponding Replicated PV Mayastor volume's NVMe target/controller ("Nexus") to be scheduled on the same Replicated PV Mayastor Node, to assist with the restoration of volume and application availability in the event of a storage node failure.
224+
225+
```
226+
kind: Pod
227+
apiVersion: v1
228+
metadata:
229+
name: fio
230+
spec:
231+
nodeSelector:
232+
openebs.io/engine: mayastor
233+
volumes:
234+
- name: ms-volume
235+
persistentVolumeClaim:
236+
claimName: ms-volume-claim
237+
containers:
238+
- name: fio
239+
image: nixery.dev/shell/fio
240+
args:
241+
- sleep
242+
- "1000000"
243+
volumeMounts:
244+
- mountPath: "/volume"
245+
name: ms-volume
246+
```
247+
248+
Verify the application.
249+
250+
**Command**
251+
252+
```
253+
oc get pods fio
254+
```
255+
256+
**Output**
257+
258+
```
259+
NAME READY STATUS RESTARTS AGE
260+
fio 1/1 Running 0 34s
261+
```
262+
263+
## See Also
264+
265+
- [Replicated PV Mayastor Installation on MicroK8s](microkubernetes.md)
266+
- [Replicated PV Mayastor Installation on Talos](talos.md)
267+
- [Provisioning NFS PVCs](../read-write-many/nfspvc.md)
268+
- [Velero Backup and Restore using Replicated PV Mayastor Snapshots - FileSystem](../backup-and-restore/velero-br-fs.md)

‎docs/sidebars.js‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -562,20 +562,25 @@ module.exports = {
562562
icon: "User"
563563
},
564564
items: [
565+
{
566+
type: "doc",
567+
id: "Solutioning/openebs-on-kubernetes-platforms/gke",
568+
label: "Google Kubernetes Engine"
569+
},
565570
{
566571
type: "doc",
567572
id: "Solutioning/openebs-on-kubernetes-platforms/microkubernetes",
568573
label: "MicroK8s"
569574
},
570575
{
571576
type: "doc",
572-
id: "Solutioning/openebs-on-kubernetes-platforms/talos",
573-
label: "Talos"
577+
id: "Solutioning/openebs-on-kubernetes-platforms/openshift",
578+
label: "OpenShift"
574579
},
575580
{
576581
type: "doc",
577-
id: "Solutioning/openebs-on-kubernetes-platforms/gke",
578-
label: "Google Kubernetes Engine"
582+
id: "Solutioning/openebs-on-kubernetes-platforms/talos",
583+
label: "Talos"
579584
}
580585
]
581586
},

0 commit comments

Comments
 (0)