Skip to content

Commit f199eb9

Browse files
committed
Add doc article
Signed-off-by: amaslennikov <[email protected]>
1 parent 2886ca6 commit f199eb9

File tree

1 file changed

+90
-0
lines changed

1 file changed

+90
-0
lines changed
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Infiniband VF GUID Static Configuration
2+
3+
SR-IOV Network Operator is able to use a static configuration file from the host filesystem to assign GUIDs to IB VFs.
4+
5+
## Prerequisites
6+
7+
- Infiniband NICs
8+
- Static GUID configuration file on the host filesystem
9+
10+
### Configuration file
11+
12+
Config file should be stored at `/etc/sriov-operator/infiniband/guids`. This location is writable across most cloud platforms.
13+
14+
VF to GUID assignment, based on this file, is ordered. VF0 takes the GUID0, VF1 takes the GUID1 etc.
15+
16+
Each PF has its own set of GUIDs.
17+
18+
Example of the config file:
19+
20+
```json
21+
[
22+
{
23+
"pci_address": "<pci_address_1>",
24+
"guids": [
25+
"02:00:00:00:00:00:00:00",
26+
"02:00:00:00:00:00:00:01"
27+
]
28+
},
29+
{
30+
"pf_guid": "<pf_guid_2>",
31+
"guidsRange": {
32+
"start": "02:00:00:00:00:aa:00:02",
33+
"end": "02:00:00:00:00:aa:00:0a"
34+
}
35+
}
36+
]
37+
```
38+
39+
Config file parameters:
40+
41+
* `pci_address` is a PCI address of a PF
42+
* `pf_guid` is a GUID of a PF. Can be obtained with `ibstat` command.
43+
* `guids` is an array of VF GUID strings
44+
* `guidsRange` is an object representing the start and end of a VF GUID range. It has two fields:
45+
* `start` is a VF GUID range start
46+
* `end` is a VF GUID range end
47+
48+
Requirements for the config file:
49+
50+
* `pci_address` and `pf_guid` cannot be set at the same time for a single device - should return an error
51+
* if the list contains multiple entries for the same device, the first one shall be taken
52+
* `rangeStart` and `rangeEnd` are both included in the range
53+
* `guids` list and range cannot be both set at the same time for a single device - should return an error
54+
* GUIDs are assigned once and not change throughout the lifecycle of the host
55+
56+
### Deploy SriovNetworkNodePolicy
57+
58+
```yaml
59+
apiVersion: sriovnetwork.openshift.io/v1
60+
kind: SriovNetworkNodePolicy
61+
metadata:
62+
name: ib-policy
63+
namespace: sriov-network-operator
64+
spec:
65+
nodeSelector:
66+
feature.node.kubernetes.io/network-sriov.capable: "true"
67+
resourceName: ib_vfs
68+
priority: 10
69+
numVfs: 2
70+
nicSelector:
71+
vendor: "15b3"
72+
rootDevices:
73+
- 0000:d8.00.0
74+
deviceType: netdevice
75+
```
76+
77+
## Verify assigned GUIDs
78+
79+
Run ip link tool to verify assigned GUIDs:
80+
81+
```bash
82+
ip link
83+
84+
...
85+
6: ibp216s0f0: <BROADCAST,MULTICAST> mtu 4092 qdisc noop state UP mode DEFAULT group default qlen 256
86+
link/infiniband 00:00:05:a9:fe:80:00:00:00:00:00:00:b8:59:9f:03:00:f9:8f:86 brd 00:ff:ff:ff:ff:12:40:1b:ff:ff:00:00:00:00:00:00:ff:ff:ff:ff
87+
vf 0 link/infiniband ... NODE_GUID 02:00:00:00:00:00:00:00, PORT_GUID 02:00:00:00:00:00:00:00, link-state enable, trust off, query_rss off
88+
vf 1 link/infiniband ... NODE_GUID 02:00:00:00:00:00:00:01, PORT_GUID 02:00:00:00:00:00:00:01, link-state enable, trust off, query_rss off
89+
```
90+

0 commit comments

Comments
 (0)