keepalived: Generate the proper route for the interface IP configured… by andrewhit · Pull Request #605 · iqiyi/dpvs

andrewhit · 2020-07-17T09:19:54Z

… within

static_ipaddress{}.

With this patch, we can get the below two expected result with the below
configuration example.

static_ipaddress {
/30 dev eth1
}

a) "dpvs-wan-ip" related route is generated.
inet /30 scope global eth1
valid_lft forever preferred_lft forever
...

b) "dpvs-wan-ip" is configured on the related interface for arp request.
inet /32 via 0.0.0.0 src dev eth1 mtu 1500 tos 0 scope link metric 0 proto auto
...

… within static_ipaddress{}. - With this patch, we can get the below two expected result with the below configuration example. static_ipaddress { dpvs-wan-ip/30 dev eth1 } a) "dpvs-wan-ip" related route is generated. inet dpvs-wan-ip/30 scope global eth1 valid_lft forever preferred_lft forever ... b) "dpvs-wan-ip" is configured on the related interface for arp request. inet dpvs-wan-network-ip/30 via 0.0.0.0 src dpvs-wan-ip dev eth1 mtu 1500 tos 0 scope link metric 0 proto auto ...

ywc689 · 2020-07-29T07:07:28Z

Test failed for the keepalived config file.

static_ipaddress {
    172.21.33.4/30 dev dpdk0
}

vrrp_instance vrrp_instance_252 {
    state MASTER
    interface dpdk0.kni
    dpdk_interface dpdk0
    virtual_router_id 252
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 12345
    }

    virtual_ipaddress {
        1.1.1.1
    }
    virtual_ipaddress_excluded {
        2.2.2.2
    }
}

logs:

169159 Jul 29 15:15:55 10 systemd: Started DPVS and VRRP High Availability Monitor.
169160 Jul 29 15:15:55 10 Keepalived[11545]: Starting VRRP child process, pid=11546
169161 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: Registering Kernel netlink reflector
169162 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: Registering Kernel netlink command channel
169163 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: Opening file '/etc/keepalived/keepalived.conf'.
169164 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: (Line 24) WARNING - interface dpdk0 for ip address 172.21.33.4/30 doesn't exist
169165 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: find dpdk interface -- dpdk0
169166 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: Non-existent interface specified in configuration
169167 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: Unregistering DPVS gratuitous ARP.
169168 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: Stopped
169169 Jul 29 15:15:55 10 Keepalived[11545]: Keepalived_vrrp exited with permanent error CONFIG. Terminating
169170 Jul 29 15:15:55 10 Keepalived[11545]: Stopped Keepalived v2.0.19 (07/29,2020), git commit v1.8.4-15-g0dd0c43

problems:

add static address 172.21.33.4/30 to dpvs interface dpdk0 failed.
virtual ip addresses 1.1.1.1/32 and 2.2.2.2/32 were not added.

andrewhit · 2020-07-29T08:25:27Z

I just tested the config within static_ipaddress{} without the config within vrrp_instance{}, for vrrp_instance{} is not used in our deployment.

static_ipaddress {
    172.21.33.4/30 dev dpdk0
}

ywc689 · 2020-07-29T09:15:01Z

I tested the patch without vrrp. The first problem still exists. Keepalived process exit abnormally immediately after startup.

169164 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: (Line 24) WARNING - interface dpdk0 for ip address 172.21.33.4/30 doesn't exist

andrewhit · 2020-07-30T07:25:20Z

169164 Jul 29 15:15:55 10 Keepalived_vrrp[11546]: (Line 24) WARNING - interface dpdk0 for ip address 172.21.33.4/30 doesn't exist

Have you tried use eth1 instead of dpdk1? Here I do not use the name, dpdk1 or dpdk0, but eth1 or eth0. And for kni interface, we use eth1_kni or eth0_kni for mlx nic.

static_ipaddress {
172.21.33.4/30 dev eth1
}

ywc689 · 2020-07-30T12:04:58Z

The dpdk interface name doesn't matter.

I followed the flow of the static address parser:
"static_addresses_handler -> alloc_saddress -> alloc_ipaddress -> if_get_by_ifname(ifname, IF_CREATE_IF_DYNAMIC)"
if_get_by_ifname failed because the specified ifname was not existing or created. I think this is why the patch didn't work.

andrewhit · 2020-07-31T02:15:14Z

The dpdk interface name doesn't matter.

I followed the flow of the static address parser:
"static_addresses_handler -> alloc_saddress -> alloc_ipaddress -> if_get_by_ifname(ifname, IF_CREATE_IF_DYNAMIC)"
if_get_by_ifname failed because the specified ifname was not existing or created. I think this is why the patch didn't work.

The reason is that when dpvs is started, we need to use dpdk-devbind --bind=igb_uio <dpdk-port> whose name is eth1 or eth0.

static_ipaddress {
    10.25.255.129/32 dev eth1
}

static_routes {
    10.0.0.0/8 via 10.25.255.1 dev eth0
}

local_address_group laddr_g1 {
   10.25.255.2 eth0
   10.25.255.3 eth0
   10.25.255.4 eth0
   10.25.255.5 eth0
   10.25.255.6 eth0
   10.25.255.7 eth0
   10.25.255.8 eth0
   10.25.255.9 eth0
   10.25.255.10 eth0
   10.25.255.11 eth0
   10.25.255.12 eth0
   10.25.255.13 eth0
   10.25.255.14 eth0
   10.25.255.15 eth0
   10.25.255.16 eth0
   10.25.255.17 eth0
}

root@n25-146-090:~# dpip addr show
inet 10.25.255.14/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.7/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.11/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.4/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.15/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.8/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.12/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.5/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.16/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.9/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.2/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.13/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.6/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.17/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.10/32 scope global eth0
     valid_lft forever preferred_lft forever
inet 10.25.255.129/32 scope global eth1
     valid_lft forever preferred_lft forever
inet 10.25.255.3/32 scope global eth0
     valid_lft forever preferred_lft forever

root@n25-146-090:~# dpip route show
inet 10.25.255.2/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 192.168.100.2/32 via 0.0.0.0 src 0.0.0.0 dev eth1 mtu 1500 tos 0 scope kni_host metric 0 proto auto
inet 10.25.255.3/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.4/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.5/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.6/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.7/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.8/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.9/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.10/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.11/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.12/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.13/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.14/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.15/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.16/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.255.17/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.25.146.90/32 via 0.0.0.0 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope kni_host metric 0 proto auto
inet 10.25.255.129/32 via 0.0.0.0 src 0.0.0.0 dev eth1 mtu 1500 tos 0 scope host metric 0 proto auto
inet 10.0.0.0/8 via 10.25.255.1 src 0.0.0.0 dev eth0 mtu 1500 tos 0 scope global metric 0 proto auto

And the below lines are observed above.

# dpip addr show
inet 10.25.255.129/32 scope global eth1
     valid_lft forever preferred_lft forever

# dpip route show
inet 10.25.255.129/32 via 0.0.0.0 src 0.0.0.0 dev eth1 mtu 1500 tos 0 scope host metric 0 proto auto

ywc689 · 2020-08-06T05:45:00Z

Please test the case where ifnames of dpvs and kni are different.

andrewhit force-pushed the dpvs-bug-22 branch from fa53421 to 06ef171 Compare July 17, 2020 09:32

andrewhit force-pushed the dpvs-bug-22 branch from 06ef171 to 158f376 Compare July 17, 2020 09:34

ywc689 approved these changes Jul 20, 2020

View reviewed changes

ywc689 added pr/to-test-codes Compile and test the patch of pr to verify if it works. pr/codes-reviewed-ok code review passed and no problem found labels Jul 20, 2020

ywc689 added pr/codes-test-failed compile problem exists or specified test cases not passed and removed pr/to-test-codes Compile and test the patch of pr to verify if it works. labels Jul 29, 2020

ywc689 removed the pr/codes-test-failed compile problem exists or specified test cases not passed label Feb 16, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

keepalived: Generate the proper route for the interface IP configured…#605

keepalived: Generate the proper route for the interface IP configured…#605
andrewhit wants to merge 1 commit intoiqiyi:develfrom
andrewhit:dpvs-bug-22

andrewhit commented Jul 17, 2020

Uh oh!

ywc689 commented Jul 29, 2020 •

edited

Loading

Uh oh!

andrewhit commented Jul 29, 2020 •

edited

Loading

Uh oh!

ywc689 commented Jul 29, 2020

Uh oh!

andrewhit commented Jul 30, 2020

Uh oh!

ywc689 commented Jul 30, 2020

Uh oh!

andrewhit commented Jul 31, 2020 •

edited

Loading

Uh oh!

ywc689 commented Aug 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

andrewhit commented Jul 17, 2020

Uh oh!

ywc689 commented Jul 29, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andrewhit commented Jul 29, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ywc689 commented Jul 29, 2020

Uh oh!

andrewhit commented Jul 30, 2020

Uh oh!

ywc689 commented Jul 30, 2020

Uh oh!

andrewhit commented Jul 31, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ywc689 commented Aug 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ywc689 commented Jul 29, 2020 •

edited

Loading

andrewhit commented Jul 29, 2020 •

edited

Loading

andrewhit commented Jul 31, 2020 •

edited

Loading