用户态套接字使用讨论

[pengtianabc]

基础信息

VMWARE下的CentOS7虚拟机环境：Linux localhost.localdomain 3.10.0-1160.el7.x86_64 #1 SMP Mon Oct 19 16:18:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

编译

修改dpdk Makefile启用IGB_UIO

make DPDK_DEBUG=y EXTRA_CFLAGS="-g -O0 -fPIC -DPRELOAD" all

准备好一个igb_uio驱动的网卡（用vfio没成功）

[root@localhost tldk]# ./dpdk/install/share/dpdk/usertools/dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:02:05.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' drv=igb_uio unused=e1000,vfio-pci

Network devices using kernel driver
===================================
0000:02:01.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens33 drv=e1000 unused=igb_uio,vfio-pci *Active*
0000:02:06.0 '82545EM Gigabit Ethernet Controller (Copper) 100f' if=ens38 drv=e1000 unused=igb_uio,vfio-pci

拓扑

vmware中的机器，其中一张虚拟网卡为nat模式，正常情况下能访问宿主机192.168.113.1的80端口；
用LD_PRELOAD不行

# start curl
export DPDK_PARAMS="--proc-type auto -w 0000:02:05.0 "
export DPDK_IP=14.18.245.164
export DPDK_IP=192.168.113.131
export DPDK_IP_MASK=24
# export DPDK_MAC=11:22:33:44:55:66
export DPDK_IP_GATEWAY=192.168.113.1
LD_PRELOAD=./libtldk.so curl -v 192.168.113.1:8000

用vfio是做客户端可以的
准备工作

# create tap0
ifconfig tap0 || tunctl -u root

# start a bridge and add tap0 into br0
brctl  addbr br0
ifconfig br0 up
ifconfig tap0 0.0.0.0 up
ifconfig br0 1.1.1.2/24 up
brctl addif br0 tap0

export DPDK_VNIC=--vdev=virtio_user0,path=/dev/vhost-net,queue_size=1024,iface=tap0
export DPDK_IP=1.1.1.1
export DPDK_IP_MASK=24
export DPDK_IP_GATEWAY=1.1.1.2

启动监听服务器

# start web server
python3 -m http.server

启动用户态套接字客户端（可用）

# start curl
LD_PRELOAD=./libtldk.so curl 1.1.1.2:8000 -v

---

启动用户态套接字服务端

LD_PRELOAD=./libtldk.so nc -lvk  1.1.1.1 8000

启动普通的nc客户端连接服务端

nc -v 1.1.1.1 8000

会出现ARP异常

07:41:35.483190 52:27:99:3b:ba:fb > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 1.1.1.1 tell 1.1.1.2, length 28
07:41:36.500283 52:27:99:3b:ba:fb > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 1.1.1.1 tell 1.1.1.2, length 28
07:41:37.928175 52:27:99:3b:ba:fb > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 1.1.1.1 tell 1.1.1.2, length 28
07:41:38.941073 52:27:99:3b:ba:fb > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 1.1.1.1 tell 1.1.1.2, length 28

手动设置arp后，只有请求没有回复，服务端没有任何信息

07:43:07.047291 52:27:99:3b:ba:fb > 52:27:99:3b:ba:fb, ethertype IPv4 (0x0800), length 74: 1.1.1.2.44210 > 1.1.1.1.8000: Flags [S], seq 2390095646, win 29200, options [mss 1460,sackOK,TS val 5267162 ecr 0,nop,wscale 7], length 0
07:43:08.052324 52:27:99:3b:ba:fb > 52:27:99:3b:ba:fb, ethertype IPv4 (0x0800), length 74: 1.1.1.2.44210 > 1.1.1.1.8000: Flags [S], seq 2390095646, win 29200, options [mss 1460,sackOK,TS val 5268167 ecr 0,nop,wscale 7], length 0
07:43:10.055486 52:27:99:3b:ba:fb > 52:27:99:3b:ba:fb, ethertype IPv4 (0x0800), length 74: 1.1.1.2.44210 > 1.1.1.1.8000: Flags [S], seq 2390095646, win 29200, options [mss 1460,sackOK,TS val 5270170 ecr 0,nop,wscale 7], length 0