Skip to content

Commit bdba4cd

Browse files
anushruti10116yuvalif
authored andcommitted
add blog post on running rook on minikube on MAC M1 host
explaining how to use docker instead of qemu Author: Anushruti Sharma <[email protected]>
1 parent 84189b7 commit bdba4cd

File tree

1 file changed

+212
-0
lines changed
  • src/en/news/blog/2025/rook-on-mac-with-docker

1 file changed

+212
-0
lines changed
Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
---
2+
title: Rook Ceph Installation on Minikube for Mac M1 with Docker Driver
3+
date: 2024-05-07
4+
author: Anushruti Sharma
5+
image: ""
6+
tags:
7+
- ceph
8+
- rook
9+
- docker
10+
- mac
11+
---
12+
13+
### Introduction
14+
15+
This article explains how to set up a test Ceph cluster that runs on a
16+
single-node Minikube cluster.
17+
18+
Docker has been chosen as the driver of the Minikube cluster on Mac M1 due to
19+
its reliability and simplicity. By choosing Docker, we avoid the complexities
20+
of virtualization, the difficulties of firewall configuration (bootpd), and the
21+
cost of x86 emulation.
22+
23+
Docker runs ARM-native containers directly. This improves performance and
24+
compatibility and lowers cost, which is important in resource-intensive systems
25+
such as Rook and Ceph.
26+
27+
#### Prerequisites
28+
29+
- MAC M1 with macOS Sonoma 14
30+
- 2 CPUs or more
31+
- 2GB of free memory
32+
- 20GB of free disk space
33+
- Internet connection
34+
35+
### Procedure
36+
37+
- Install docker
38+
39+
```
40+
brew install docker
41+
brew install colima
42+
colima start
43+
```
44+
45+
- Install and start Minikube
46+
47+
```
48+
brew install minikube
49+
minikube start --disk-size=20g --driver docker
50+
```
51+
52+
- Install kubectl on your host machine
53+
54+
```
55+
curl -LO "https://dl.k8s.io/release/v1.26.1/bin/darwin/arm64/kubectl"
56+
chmod +x kubectl
57+
sudo mv kubectl /usr/local/bin/
58+
```
59+
60+
- Because the M1 Mac is ARM-based, it is not possible to attach /dev/sd*
61+
devices or /dev/vd* devices. In this situation, we would normally default to
62+
the use of /dev/loop devices, but Ceph does not permit the use of /dev/loop
63+
devices. Instead, we will use the network block device /dev/nbd0.
64+
65+
```
66+
minikube ssh
67+
sudo mkdir /mnt/disks
68+
69+
# Create an empty file of size 10GB to mount disk as ceph osd
70+
71+
sudo dd if=/dev/zero of=/mnt/disks/mydisk.img bs=1M count=10240
72+
sudo apt update
73+
sudo apt upgrade
74+
sudo apt-get install qemu-utils
75+
76+
# List the nbd devices
77+
78+
lsblk | grep nbd
79+
80+
# If you are unable to see the nbd device, load the NBD (Network Block Device)
81+
kernel module.
82+
83+
sudo modprobe nbd max_part=8
84+
85+
# To bind nbd device to the file
86+
# Note: Please check there is no necessary data in /dev/nbdx, otherwise back up
87+
that data.
88+
89+
sudo qemu-nbd --format raw -c /dev/nbd0 /mnt/disks/mydisk.img
90+
```
91+
92+
- Verify the size of the nbd device by using lsblk
93+
94+
```
95+
lsblk | grep nbd0
96+
```
97+
98+
- Clone the Rook repository to your host machine.
99+
100+
```
101+
git clone https://github.com/rook/rook.git
102+
```
103+
104+
- Move into the rook/deploy/examples/ directory.
105+
106+
```
107+
cd rook/deploy/examples/
108+
```
109+
110+
- Deploy the Rook operator.
111+
112+
```
113+
kubectl create -f crds.yaml -f common.yaml -f operator.yaml
114+
```
115+
116+
- Verify that the rook-ceph-operator is in a running state before proceeding.
117+
118+
```
119+
kubectl get pods -n rook-ceph
120+
```
121+
122+
- In cluster-test.yaml, make necessary changes to the storage section with the
123+
device selection:
124+
125+
```
126+
storage:
127+
useAllNodes: false
128+
useAllDevices: false
129+
nodes:
130+
- name: minikube # node name of minikube node
131+
devices:
132+
- name: /dev/nbd0 # device name being used
133+
allowDeviceClassUpdate: true
134+
allowOsdCrushWeightUpdate: false
135+
```
136+
137+
- Create the Ceph cluster.
138+
139+
```
140+
kubectl create -f cluster-test.yaml
141+
```
142+
143+
- Verify that the cluster is running by checking the pods' status in the
144+
rook-ceph namespace
145+
146+
```
147+
kubectl -n rook-ceph get pod
148+
```
149+
150+
- If the rook-ceph-mon, rook-ceph-mgr, or rook-ceph-osd pods are not created,
151+
refer to [Ceph common
152+
issues](https://rook.io/docs/rook/latest/Troubleshooting/ceph-common-issues/)
153+
for more information.
154+
155+
- To verify that the cluster is in a healthy state, connect to the Rook
156+
Toolbox.
157+
158+
```
159+
kubectl create -f toolbox.yaml
160+
```
161+
162+
- Wait for the toolbox pod to download its container and then to arrive in a
163+
running state:
164+
165+
```
166+
kubectl -n rook-ceph rollout status deploy/rook-ceph-tools
167+
```
168+
169+
- After the rook-ceph-tools pod is running, you can connect to it with the
170+
following command:
171+
172+
```
173+
kubectl -n rook-ceph exec -it deploy/rook-ceph-tools -- bash
174+
```
175+
176+
- Run the `ceph status` command and ensure the following:
177+
178+
- All mons are in quorum
179+
- A mgr is active
180+
- At least 1 OSDs should be up and in
181+
- If the health status is not HEALTH_OK, investigate the warnings or errors
182+
183+
```
184+
bash-5.1$ ceph -s
185+
cluster:
186+
id: f89dd5e5-e2bb-44e8-8969-659f0fc9dc55
187+
health: HEALTH_OK
188+
189+
services:
190+
mon: 1 daemons, quorum a (age 7m)
191+
mgr: a(active, since 5m)
192+
osd: 1 osds: 1 up (since 6m), 1 in (since 6m)
193+
194+
data:
195+
pools: 1 pools, 1 pgs
196+
objects: 2 objects, 449 KiB
197+
usage: 27 MiB used, 10 GiB / 10 GiB avail
198+
pgs: 1 active+clean
199+
```
200+
201+
If the cluster is not healthy, refer to the [Ceph common
202+
issues](https://rook.io/docs/rook/latest/Troubleshooting/ceph-common-issues/)
203+
for potential solutions.
204+
205+
Footnote:
206+
207+
Thanks Yuval Lifshitz for providing all the support and guidance to write this
208+
article.
209+
210+
References:
211+
212+
https://rook.io/docs/rook/latest/Getting-Started/quickstart/

0 commit comments

Comments
 (0)