Skip to content

Commit 7878391

Browse files
committed
Add kalray plugin to configure DPU
This plugin allows to create raid, logical volume store and logical volumes on the Kalray DPU. It also allows the deletion of a volume. Currently restrications are due to the Kalray DPU. Next generation of the DPU will allow more volumes and won't be restricted on volume name. See the REAMDE for details. Signed-off-by: Guillaume <[email protected]>
1 parent e9415e0 commit 7878391

File tree

5 files changed

+396
-0
lines changed

5 files changed

+396
-0
lines changed

Diff for: README.md

+98
Original file line numberDiff line numberDiff line change
@@ -194,6 +194,104 @@ $ xe host-call-plugin host-uuid=<uuid> plugin=hyperthreading.py fn=get_hyperthre
194194
true
195195
```
196196

197+
## XCP-ng Kalray DPU
198+
199+
A xapi plugin to get information about raids, logical volume store (LVS) and
200+
devices that are present on the Kalray DPU. It also allow the management of
201+
logical volumes (LV): creation and deletion. Parameters depends of the name of
202+
the command some are always available:
203+
- *username*: username to use to connect to DPU (required)
204+
- *password*: password to connect to DPU (required)
205+
- *server*: IP of the server for configuring the DPU (default: localhost)
206+
- *port*: Port to use (default: 8080)
207+
- *timeout*: timeout in second (default: 60.0)
208+
209+
### Command details
210+
211+
- Currently the Kalray DPU is still in developpement and there are some
212+
restrictions:
213+
- To be able to expose virtual functions the Kalray poller expects
214+
specific names for the logical volume store and for the volume. It
215+
depends of the configuration used in `/etc/kalray/0000:XX:00.1.conf`.
216+
- By default the logical volume store name **must be** `lvs`.
217+
- By default the volume must start with `volume_`.
218+
- With the current DPU only four virtual functions (and so only four NVMe
219+
disks) can be created and so you can only use the following name:
220+
- `volume_09`
221+
- `volume_10`
222+
- `volume_11`
223+
- `volume_12`
224+
- Only volumes can be deleted.
225+
226+
#### Block devices
227+
228+
##### Get the list of devices on the Kalray DPU
229+
```
230+
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=get_devices \
231+
args:username=<username> args:password=<password>
232+
{"status": "ok", "output": [{"name": "HotInNvmeWDS500AFY0-22050C800415n1", "aliases": [], "product_name": "NVMe disk", "block_size": 512, "num_blocks": 976773168, "uuid": "e8238fa6-bf53-0001-001b-448b45afa6a7", "assigned_rate_limits": {"rw_ios_per_sec": 0, "rw_mbytes_per_sec": 0, "r_mbytes_per_sec": 0, "w_mbytes_per_sec": 0}, "claimed": false, "zoned": false, "supported_io_types": {"read": true, "write": true, "unmap": true, "write_zeroes": true, "flush": true, "reset": true, "nvme_admin": true, "nvme_io": true}, "driver_specific": {"nvme": [{"pci_address": "0000:00:00.0", "trid": {"trtype": "PCIe", "traddr": "0000:00:00.0"}, "ctrlr_data": {"cntlid": 8224, "vendor_id": "0x15b7", "model_number": "WDS500G1X0E-00AFY0", "serial_number": "22050C800415", "firmware_revision": "614900WD", "subnqn": "nqn.2018-01.com.wdc:nguid:E8238FA6BF53-0001-001B448B45AFA6A7", "oacs": {"security": 1, "format": 1, "firmware": 1, "ns_manage": 0}, "multi_ctrlr": false, "ana_reporting": false}, "vs": {"nvme_version": "1.4"}, "ns_data": {"id": 1, "can_share": false}, "security": {"opal": false}}], "mp_policy": "active_passive"}}]}
233+
```
234+
235+
#### RAID
236+
237+
##### Create a raid on the Kalray DPU
238+
- Supported RAID are raid0, raid1 and raid10
239+
```
240+
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=raid_create \
241+
args:username=<username> args:password=<password> \
242+
args:base_bdevs=HotInNvmeWDS500AFY0-22050C800415n1,HotInNvmeWDS500AFY0-22050C800378n1 \
243+
args:raid_name=raid0 \
244+
args:raid_level=raid0
245+
true
246+
```
247+
248+
##### Get the list of raids on the Kalray DPU
249+
```
250+
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=get_raids \
251+
args:username=<username> args:password=<password>
252+
["raid0"]
253+
```
254+
255+
#### Logical Volume Store (LVS)
256+
##### Create an LVS on the Kalray DPU
257+
```
258+
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=lvs_create \
259+
args:username=<username> args:password=<password> \
260+
args:lvs_name=lvs \
261+
args:bdev_name=raid0
262+
"6fb90332-56e4-4d03-aa6a-f858a2c2ca97"
263+
```
264+
265+
##### Get the list of LVS on the Kalray DPU
266+
```
267+
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=get_lvs \
268+
args:username=<username> args:password=<password>
269+
[{"uuid": "6fb90332-56e4-4d03-aa6a-f858a2c2ca97", "name": "lvs", "passive": false, "base_bdev": "raid0", "total_data_clusters": 29804, "free_clusters": 29772, "block_size": 512, "cluster_size": 33554432}]
270+
```
271+
272+
#### Logical Volume (LVOL)
273+
##### Create a new logical volume
274+
```
275+
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=lvol_create \
276+
args:username=<username> args:password=<password> \
277+
args:lvol_name=volume_09 \
278+
args:lvol_size_mb=1048576 \
279+
args:lvs_name=lvs
280+
"6c84b44c-a61b-41a4-8b19-32ab643b57d9"
281+
```
282+
283+
##### Delete a logical volume
284+
- The name of the volume to be deleted is not the same than the one used to
285+
create it. You need to prepend the name of the logical volume store as shown
286+
in the example:
287+
288+
```
289+
$ xe host-call-plugin host-uuid=<uuid> plugin=kalray_dpu.py fn=lvol_delete \
290+
args:username=<username> args:password=<password> \
291+
args:lvol_name=lvs/volume_09
292+
true
293+
```
294+
197295
## Tests
198296

199297
To run the plugins' unit tests you'll need to install `pytest`, `pyfakefs` and `mock`.

Diff for: SOURCES/etc/xapi.d/plugins/kalray_dpu.py

+150
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
#!/usr/bin/python3
2+
"""XAPI plugin to manage Kalray DPU."""
3+
4+
import json
5+
import XenAPIPlugin # pylint: disable=import-error
6+
7+
from kalray.acs.spdk.rpc.client import HTTPJSONRPCClient, JSONRPCException # pylint: disable=import-error
8+
from xcpngutils import error_wrapped
9+
10+
class KalrayCmd:
11+
"""Describe a command to be ran on the Kalray DPU."""
12+
13+
def __init__(self, rpc_name: str, updates: dict):
14+
self.server = 'localhost'
15+
self.port = 8080
16+
self.username = None
17+
self.password = None
18+
self.timeout = 60.0
19+
self.rpc_name = rpc_name
20+
self.rpc_params = {} # will be updated using add_rpc_params
21+
22+
for k, v in updates.items():
23+
if hasattr(self, k):
24+
setattr(self, k, v)
25+
26+
# Check that username & password are well set
27+
if self.username is None:
28+
raise XenAPIPlugin.Failure("-1", ["'username' is required"])
29+
if self.password is None:
30+
raise XenAPIPlugin.Failure("-1", ["'password' is required"])
31+
32+
def add_rpc_params(self, key, value):
33+
"""Adds a parameter that will be passed to the RPC."""
34+
self.rpc_params[key] = value
35+
36+
def call_rpc(self):
37+
"""Do the RPC call."""
38+
try:
39+
client = HTTPJSONRPCClient(
40+
self.server,
41+
self.port,
42+
self.timeout,
43+
self.username,
44+
self.password,
45+
log_level="ERROR")
46+
message = client.call(self.rpc_name, self.rpc_params)
47+
except JSONRPCException as exc:
48+
raise XenAPIPlugin.Failure("-1", [exc.message])
49+
50+
return json.dumps(message)
51+
52+
@error_wrapped
53+
def get_devices(_session, args):
54+
"""Get the list of devices available on the Kalray DPU."""
55+
kc = KalrayCmd("bdev_get_bdevs", args)
56+
return kc.call_rpc()
57+
58+
@error_wrapped
59+
def get_raids(_session, args):
60+
"""Get the list of raids available on the Kalray DPU."""
61+
kc = KalrayCmd("bdev_raid_get_bdevs", args)
62+
kc.add_rpc_params("category", "all")
63+
return kc.call_rpc()
64+
65+
@error_wrapped
66+
def get_lvs(_session, args):
67+
"""Get the list of logical volume stores available on the Kalray DPU."""
68+
kc = KalrayCmd("bdev_lvol_get_lvstores", args)
69+
return kc.call_rpc()
70+
71+
@error_wrapped
72+
def raid_create(_session, args):
73+
"""Create a raid."""
74+
kc = KalrayCmd("bdev_raid_create", args)
75+
try:
76+
raid_name = args["raid_name"]
77+
raid_level = args["raid_level"]
78+
base_bdevs = args["base_bdevs"].split(',')
79+
except KeyError as msg:
80+
raise XenAPIPlugin.Failure("-1", [f"Key {msg} is missing"])
81+
82+
# Check supported raids
83+
if raid_level not in ["raid0", "raid1", "raid10"]:
84+
raise XenAPIPlugin.Failure("-1", ["Only raid0, raid1 and raid10 are supported"])
85+
86+
kc.add_rpc_params("name", raid_name)
87+
kc.add_rpc_params("raid_level", raid_level)
88+
kc.add_rpc_params("base_bdevs", base_bdevs)
89+
kc.add_rpc_params("strip_size_kb", 128)
90+
kc.add_rpc_params("persist", True)
91+
kc.add_rpc_params("split_dp", True)
92+
return kc.call_rpc()
93+
94+
@error_wrapped
95+
def lvs_create(_session, args):
96+
"""Create a logical volume store."""
97+
kc = KalrayCmd("bdev_lvol_create_lvstore", args)
98+
try:
99+
lvs_name = args["lvs_name"]
100+
bdev_name = args["bdev_name"]
101+
except KeyError as msg:
102+
raise XenAPIPlugin.Failure("-1", [f"Key {msg} is missing"])
103+
104+
kc.add_rpc_params("lvs_name", lvs_name)
105+
kc.add_rpc_params("bdev_name", bdev_name)
106+
107+
return kc.call_rpc()
108+
109+
@error_wrapped
110+
def lvol_create(_session, args):
111+
"""Create a new lvol on the Kalray DPU."""
112+
kc = KalrayCmd("bdev_lvol_create", args)
113+
114+
try:
115+
lvol_name = args["lvol_name"]
116+
lvol_size = int(args["lvol_size_mb"])
117+
lvs_name = args["lvs_name"]
118+
except KeyError as msg:
119+
raise XenAPIPlugin.Failure("-1", [f"Key {msg} is missing"])
120+
except ValueError as msg:
121+
raise XenAPIPlugin.Failure("-1", [f"Wrong size: {msg}"])
122+
123+
kc.add_rpc_params("lvol_name", lvol_name)
124+
kc.add_rpc_params("size", lvol_size)
125+
kc.add_rpc_params("lvs_name", lvs_name)
126+
return kc.call_rpc()
127+
128+
@error_wrapped
129+
def lvol_delete(_session, args):
130+
"""Delete the lvol passed as parameter on the Kalray DPU if exists."""
131+
kc = KalrayCmd("bdev_lvol_delete", args)
132+
133+
try:
134+
lvol_name = args["lvol_name"]
135+
except KeyError as msg:
136+
raise XenAPIPlugin.Failure("-1", [f"Key {msg} is missing"])
137+
138+
kc.add_rpc_params("name", lvol_name)
139+
return kc.call_rpc()
140+
141+
if __name__ == "__main__":
142+
XenAPIPlugin.dispatch({
143+
"get_devices": get_devices,
144+
"get_raids": get_raids,
145+
"get_lvs": get_lvs,
146+
"raid_create": raid_create,
147+
"lvs_create": lvs_create,
148+
"lvol_create": lvol_create,
149+
"lvol_delete": lvol_delete,
150+
})

Diff for: tests/conftest.py

+3
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@
55
import mocked_configparser
66
import mocked_xen_api_plugin
77
import mocked_yum
8+
import mocked_kalray_rpc
89

910
def pytest_configure():
1011
pytest.plugins_lock_file = '/var/lib/xcp-ng-xapi-plugins/pytest.lock'
@@ -18,6 +19,8 @@ def pytest_configure():
1819
# Mock yum globally, module is not necessarily present on the system.
1920
sys.modules['yum'] = mocked_yum
2021

22+
sys.modules['kalray.acs.spdk.rpc.client'] = mocked_kalray_rpc
23+
2124
sys.path.append(str(pathlib.Path(__file__).parent.resolve()) + '/../SOURCES/etc/xapi.d/plugins')
2225

2326
pytest_plugins = ("pyfakefs",)

Diff for: tests/mocked_kalray_rpc.py

+55
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
class HTTPJSONRPCClient(object):
2+
def __init__(self, addr, port=None, timeout=60.0, user='admin', password='admin', **kwargs):
3+
pass
4+
5+
def call(self, method, params=None):
6+
"""We will juste check that parameters are ok."""
7+
parameters = {
8+
"bdev_get_bdevs": {
9+
"required": [],
10+
"optional": ['name', 'timeout'],
11+
},
12+
"bdev_raid_get_bdevs": {
13+
"required": ['category'],
14+
"optional": [],
15+
},
16+
"bdev_lvol_get_lvstores": {
17+
"required": [],
18+
"optional": ['uuid', 'lvs_name'],
19+
},
20+
"bdev_raid_create": {
21+
"required": ['name', 'strip_size_kb', 'raid_level', 'base_bdevs'],
22+
"optional": ['persist', 'split_dp'],
23+
},
24+
"bdev_lvol_create_lvstore": {
25+
"required": ['bdev_name', 'lvs_name'],
26+
"optional": ['cluster_sz', 'clear_method', 'num_md_pages_per_cluster_ratio'],
27+
},
28+
"bdev_lvol_create": {
29+
"required": ['lvol_name'],
30+
"optional": ['size', 'size_in_mib', 'thin_provision', 'uuid', 'lvs_name', 'clear_method'],
31+
},
32+
"bdev_lvol_delete": {
33+
"required": ['name'],
34+
"optional": [],
35+
},
36+
}
37+
38+
# Check that method is mocked
39+
try:
40+
p = parameters[method]
41+
except KeyError:
42+
assert False, f"{method} is not mocked"
43+
44+
# Check that required parameters are given
45+
for k in p['required']:
46+
assert k in params, f"Required parameter '{k}' is missing for {method}"
47+
48+
# Check that params passed to method are valid
49+
for k in params:
50+
assert k in p['required'] or k in p['optional'], f"Invalid parameter '{k}' for {method}"
51+
52+
53+
class JSONRPCException(BaseException):
54+
def __init__(self, message):
55+
assert False, "Mock me!"

0 commit comments

Comments
 (0)