fix-nfs (#2759)

* fix-nfs

* permissive policy update

* add cleanup scripts

* add cleanup scripts

Author: s1saurabh

OracleDatabase/RAC/OracleDNSServer/README.md

@@ -38,8 +38,8 @@ docker network create --driver=bridge --subnet=192.168.17.0/24 rac_priv1_nw
 ```
 **Note:** You can change the subnet according to your environment.
 
-### Running RAC DNS server container
-Execute following command to create the container on Docker Host:
+## Running RAC DNS server container
+### Execute following command to create the container on Docker Host
 
 ```bash
 docker create --hostname racdns \
@@ -61,7 +61,7 @@ docker network connect rac_priv1_nw --ip 192.168.17.25 rac-dnsserver
 docker start rac-dnsserver
 ```
 
-Execute following command to create the container on Podman Host:
+### Execute following command to create the container on Podman Host
 
 ```bash
 podman create --hostname racdns \
@@ -99,4 +99,4 @@ you should see the following in docker logs output:
 #################################################
 DNS Server IS READY TO USE!
 #################################################
-```
+```

OracleDatabase/RAC/OracleRACStorageServer/README.md

@@ -136,6 +136,12 @@ mkdir -p /scratch/stage/rac-storage/$ORACLE_DBNAME
 rm -rf /scratch/stage/rac-storage/$ORACLE_DBNAME/asm_disk0*
 ```
 
+If SELinux is enabled on Podman Host (you can check by running `sestatus` command), then execute below to make SELinux policy as `permissive` and reboot the host machine. This will allow permissions to write to `asm-disks*` in the `/oradata` folder inside the podman containers-
+```bash
+sed -i 's/^SELINUX=.*/SELINUX=permissive/' /etc/selinux/config
+reboot
+```
+
 Execute following command to create the container:
 
 ```bash

OracleDatabase/RAC/OracleRealApplicationClusters/README.md

@@ -623,7 +623,7 @@ You must install and configure [Podman release 4.0.2](https://docs.oracle.com/en
   
 - You can check the details on [Oracle Linux and Unbreakable Enterprise Kernel (UEK) Releases](https://blogs.oracle.com/scoter/post/oracle-linux-and-unbreakable-enterprise-kernel-uek-releases)
 
-- You do not need to execute step 2 in this section to create and enable `Podman-rac-cgroup.service` when we are running Oracle Linux 8 with Unbreakable Enterprise Kernel R7.
+- You do not need to execute step 2 in this section to create and enable `podman-rac-cgroup.service` when we are running Oracle Linux 8 with Unbreakable Enterprise Kernel R7.
 
 **IMPORTANT:** Completing prerequisite steps is a requirement for successful configuration.
 

OracleDatabase/RAC/OracleRealApplicationClusters/samples/racdockercompose/README.md

@@ -14,6 +14,7 @@ Once you have built your Oracle RAC container image, you can create a Oracle RAC
     - [Section 4.1: Add Additional Node in Existing Oracle RAC Cluster with Block Devices](#section-41-add-additional-node-in-existing-oracle-rac-cluster-with-block-devices)
     - [Section 4.2: Add Additional Node in Existing Oracle RAC Cluster with NFS Volume](#section-42-add-additional-node-in-existing-oracle-rac-cluster-with-nfs-volume)
   - [Section 5: Connect to the RAC container](#connect-to-the-rac-container)
+  - [Cleanup RAC Environment](#cleanup-rac-environment)
   - [Copyright](#copyright)
 
 ## Section 1 : Prerequisites for RAC Database on Docker with Docker Compose
@@ -451,6 +452,38 @@ docker exec -i -t racnoded1 /bin/bash
 
 If the install fails for any reason, log in to container using the above command and check `/tmp/orod.log`. You can also review the Grid Infrastructure logs located at `$GRID_BASE/diag/crs` and check for failure logs. If the failure occurred during the database creation then check the database logs.
 
+## Cleanup RAC Environment
+Below commands can be executed to cleanup above RAC Environment -
+
+### Cleanup RAC based on Block Devices
+```bash
+#----Cleanup RAC Containers-----
+docker rm -f racnoded1 racnoded2 rac-dnsserver racnodedc1-cman 
+#----Cleanup Disks--------------
+dd if=/dev/zero of=/dev/oracleoci/oraclevde  bs=8k count=10000 status=progress && dd if=/dev/zero of=/dev/oracleoci/oraclevdd  bs=8k count=10000 status=progress
+#----Cleanup Files and Folders--
+rm -rf /opt/containers /opt/.secrets
+#----Cleanup Docker Networks--
+docker network rm -f rac_pub1_nw rac_pzriv1_nw
+#----Cleanup Docker Images--
+docker rmi -f oracle/rac-dnsserver:latest oracle/database-rac:19.3.0 oracle/client-cman:19.3.0
+```
+
+### Cleanup RAC based on NFS Storage Devices
+```bash
+#----Cleanup RAC Containers-----
+docker rm -f racnoded1 racnoded2 rac-dnsserver racnode-storage racnodedc1-cman 
+#----Cleanup Files and Folders--
+rm -rf /opt/containers /opt/.secrets
+export ORACLE_DBNAME=ORCLCDB
+rm -rf /docker_volumes/asm_vol/$ORACLE_DBNAME/asm_disk0*
+#----Cleanup Docker Volumes---
+docker volume -f racstorage
+#----Cleanup Docker Networks--
+docker network rm -f rac_pub1_nw rac_pzriv1_nw
+#----Cleanup Docker Images--
+docker rmi -f oracle/rac-dnsserver:latest oracle/rac-storage-server:19.3.0 oracle/database-rac:19.3.0 oracle/client-cman:19.3.0
+```
 
 ## Copyright
 

OracleDatabase/RAC/OracleRealApplicationClusters/samples/racpodmancompose/README.md

@@ -14,6 +14,7 @@ Once you have built your Oracle RAC container image, you can create a Oracle RAC
     - [Section 4.1: Add Additional Node in Existing Oracle RAC Cluster with Block Devices](#section-41-add-additional-node-in-existing-oracle-rac-cluster-with-block-devices)
     - [Section 4.2: Add Additional Node in Existing Oracle RAC Cluster with NFS Volume](#section-42-add-additional-node-in-existing-oracle-rac-cluster-with-nfs-volume)
   - [Section 5: Connect to the RAC container](#connect-to-the-rac-container)
+  - [Cleanup RAC Environment](#cleanup-rac-environment)
   - [Copyright](#copyright)
 
 ## Section 1 : Prerequisites for RAC Database on Podman with Podman Compose
@@ -92,6 +93,12 @@ In order to setup Oracle RAC on Podman with Oracle RAC Storage Container with Po
 yum -y install nfs-utils
 ```
 
+If SELinux is enabled on Podman Host (you can check by running `sestatus` command), then execute below to make SELinux policy as `permissive` and reboot host machine. This will allow permissions to write to `asm-disks*` in the `/oradata` folder inside the podman containers-
+```bash
+sed -i 's/^SELINUX=.*/SELINUX=permissive/' /etc/selinux/config
+reboot
+```
+
 Lets identify necessary variables to export that will be used by `podman-compose.yml` file later. Below is one example of exporting necessary variables related to docker network, DNS container, Storage Container, RAC Container and CMAN container discussed in this repo.
 ```bash
 export HEALTHCHECK_INTERVAL=30s
@@ -241,15 +248,6 @@ mkdir -p /scratch/stage/rac-storage/$ORACLE_DBNAME
 rm -rf /scratch/stage/rac-storage/ORCLCDB/asm_disk0*
 ```
 
-```bash
-#----------Create NFS volume--------------
-podman volume create --driver local \
---opt type=nfs \
---opt   o=addr=192.168.17.80,rw,bg,hard,tcp,vers=3,timeo=600,rsize=32768,wsize=32768,actimeo=0 \
---opt device=192.168.17.80:/oradata \
-racstorage
-```
-
 After copying compose file, you can bring up DNS Container, Storage Container, RAC Container and CMAN container by following below commands-
 ```bash
 #---------Bring up DNS------------
@@ -278,6 +276,16 @@ Export list for racnode-storage:
 #################################################
 ```
 
+```bash
+#----------Create NFS volume--------------
+podman volume create --driver local \
+--opt type=nfs \
+--opt   o=addr=192.168.17.80,rw,bg,hard,tcp,vers=3,timeo=600,rsize=32768,wsize=32768,actimeo=0 \
+--opt device=192.168.17.80:/oradata \
+racstorage
+```
+
+
 ```bash
 #-----Bring up racnode1----------
 podman-compose --podman-run-args="-t -i --systemd=always --cpuset-cpus 0-1 --memory 16G --memory-swap 32G" up -d ${RACNODE1_CONTAINER_NAME} && \
@@ -444,6 +452,38 @@ podman exec -i -t racnodep1 /bin/bash
 
 If the install fails for any reason, log in to container using the above command and check `/tmp/orod.log`. You can also review the Grid Infrastructure logs located at `$GRID_BASE/diag/crs` and check for failure logs. If the failure occurred during the database creation then check the database logs.
 
+## Cleanup RAC Environment
+Below commands can be executed to cleanup above RAC Environment -
+
+### Cleanup RAC based on Block Devices
+```bash
+#----Cleanup RAC Containers-----
+podman rm -f racnodep1 racnodep2 rac-dnsserver racnodepc1-cman 
+#----Cleanup Disks--------------
+dd if=/dev/zero of=/dev/oracleoci/oraclevde  bs=8k count=10000 status=progress && dd if=/dev/zero of=/dev/oracleoci/oraclevdd  bs=8k count=10000 status=progress
+#----Cleanup Files and Folders--
+rm -rf /opt/containers /opt/.secrets
+#----Cleanup Docker Networks--
+podman network rm -f rac_pub1_nw rac_zriv1_nw
+#----Cleanup Docker Images--
+podman rmi -f localhost/oracle/rac-dnsserver:latest localhost/oracle/database-rac:21.3.0-21.13.0 localhost/oracle/client-cman:21.3.0
+```
+
+### Cleanup RAC based on NFS Storage Devices
+```bash
+#----Cleanup RAC Containers-----
+podman rm -f racnodep1 racnodep2 rac-dnsserver racnode-storage racnodepc1-cman 
+#----Cleanup Files and Folders--
+rm -rf /opt/containers /opt/.secrets
+export ORACLE_DBNAME=ORCLCDB
+rm -rf /scratch/stage/rac-storage/ORCLCDB/asm_disk0*
+#----Cleanup Docker Volumes---
+podman volume -f racstorage
+#----Cleanup Docker Networks--
+podman network rm -f rac_pub1_nw rac_priv1_nw
+#----Cleanup Docker Images--
+podman rmi -f localhost/oracle/rac-dnsserver:latest localhost/oracle/rac-storage-server:latest localhost/oracle/database-rac:21.3.0-21.13.0 localhost/oracle/client-cman:21.3.0
+```
 
 ## Copyright