Skip to content

Commit e6a58be

Browse files
committed
Add cpu_balancer to template.yaml
1 parent f5f80ba commit e6a58be

File tree

3 files changed

+497
-0
lines changed

3 files changed

+497
-0
lines changed

server/agent_config/README-CH.md

+107
Original file line numberDiff line numberDiff line change
@@ -4355,6 +4355,113 @@ inputs:
43554355

43564356
TODO
43574357

4358+
### CPU 均衡器 {#inputs.ebpf.cpu_balancer}
4359+
4360+
**标签**:
4361+
4362+
<mark>agent_restart</mark>
4363+
4364+
**FQCN**:
4365+
4366+
`inputs.ebpf.cpu_balancer`
4367+
4368+
4369+
**默认值**:
4370+
```yaml
4371+
inputs:
4372+
ebpf:
4373+
cpu_balancer:
4374+
- nic_name: ""
4375+
ring_size: 4096
4376+
nic_rx_cpus: []
4377+
xdp_cpus: []
4378+
mode: 0
4379+
```
4380+
4381+
**模式**:
4382+
| Key | Value |
4383+
| ---- | ---------------------------- |
4384+
| Type | dict |
4385+
4386+
**详细描述**:
4387+
4388+
通过RSS和XDP技术的结合,网络流量可以分配到不同的CPU核心,从而实现更高效的网络数据包处理。整个从网卡
4389+
(NIC)到最终处理的路径如下:
4390+
4391+
`[NIC] --(interrupt)--> [RSS receive CPU] --(XDP CPUMAP)--> [Softirq CPU1, CPU2 ...]`
4392+
4393+
当网卡(NIC)接收到网络数据包时,它会触发一个中断信号。这是一个硬件级别的中断,用于通知系统有数据包到达。
4394+
然后,使用XDP CPUMAP将数据包分配到其他CPU核心进行进一步处理。XDP CPUMAP是一种机制,用于将网络数据包从接收
4395+
CPU映射并分发到多个处理CPU,从而实现更高效的软中断处理。
4396+
当网络接口接收到诸如 GRE、Double VLAN(QinQ)或 VXLAN 等封装数据包时,由于网卡的 RSS(接收端扩展)无法将其均
4397+
匀分配到各个 CPU 上,XDP CPUMAP 可以有效缓解这一问题。我们可以使用 `top` 命令查看每个 CPU 的软中断使用情况,
4398+
以及使用 `rxtxcpu` 命令查看每个 CPU 的数据收集计数,从而判断 CPU 数据处理是否存在不均衡的情况。
4399+
4400+
`rxtxcpu` 命令:
4401+
```
4402+
# rxtxcpu -d rx <nic-name>
4403+
110 packets captured on cpu0.
4404+
93 packets captured on cpu1.
4405+
112 packets captured on cpu2.
4406+
77 packets captured on cpu3.
4407+
853 packets captured on cpu4.
4408+
53 packets captured on cpu5.
4409+
...
4410+
```
4411+
可以查看每个 CPU 接收数据包的统计信息。如果某个 CPU 的计数明显高于其他 CPU,这表明 CPU 处理存在不均衡的情况。
4412+
4413+
配置项:
4414+
- nic_name:网络接口名称,默认值:""
4415+
- ring_size:接收(RX)环形缓冲区大小,默认值:4096
4416+
- nic_rx_cpus:处理网卡(NIC)接收的网络数据的CPU列表,这些数据由物理中断触发。由于网卡(NIC)包含多个物理核心,
4417+
每个物理核心有多个逻辑核心,我们选择网卡所在NUMA节点的物理核心。可以按照以下方式进行操作:
4418+
```
4419+
# deepflow-ebpfctl nicinfo --interface=p1p1
4420+
Device: p1p1
4421+
Address: 0000:41:00.0
4422+
Driver: i40e
4423+
Rx-Channels: 64
4424+
Tx-Channels: 64
4425+
RX-Ring-Size: 512
4426+
TX-Ring-Size: 512
4427+
PROMISC: 0
4428+
NumaNode: 1
4429+
4430+
# deepflow-ebpfctl cpu_layout show
4431+
cores = [0, 1, 2, 3, 4, 5]
4432+
sockets = [0, 1]
4433+
4434+
Socket 0 Socket 1
4435+
------------ ------------
4436+
Core 0 [0, 12] [1, 13]
4437+
Core 1 [2, 14] [3, 15]
4438+
Core 2 [4, 16] [5, 17]
4439+
Core 3 [6, 18] [7, 19]
4440+
Core 4 [8, 20] [9, 21]
4441+
Core 5 [10, 22] [11, 23]
4442+
```
4443+
从上述信息我们得知,网卡 p1p1 位于 NUMA 节点 1。我们需要从 Socket 1 中选择核心,例如:选择 Core 0 和 Core 1。
4444+
从 Core 0 中可以选择逻辑核心 1 或 13,从 Core 1 中可以选择逻辑核心 3 或 15。配置可以设置为 [13,15]。
4445+
默认值:[]
4446+
- xdp_cpus: XDP CPU 分发列表,确保分发的 CPU 不与 nic_rx_cpus 重叠,默认值:[]
4447+
- mode: CPU 均衡模式,默认值:0
4448+
4449+
样例:
4450+
```yaml
4451+
inputs:
4452+
ebpf:
4453+
cpu_balancer:
4454+
- nic_name: eth0
4455+
ring_size: 4096
4456+
nic_rx_cpus: [1,3]
4457+
xdp_cpus: [5,7,9,11]
4458+
mode: 0
4459+
- nic_name: eth1
4460+
ring_size: 4096
4461+
nic_rx_cpus: [13,15]
4462+
xdp_cpus: [17,18,19,21]
4463+
mode: 0
4464+
```
43584465
## 资源 {#inputs.resources}
43594466

43604467
### 推送间隔 {#inputs.resources.push_interval}

server/agent_config/README.md

+120
Original file line numberDiff line numberDiff line change
@@ -4489,6 +4489,126 @@ inputs:
44894489

44904490
Set the maximum value of hash table entries for thread/coroutine tracking sessions.
44914491

4492+
### CPU Balancer {#inputs.ebpf.cpu_balancer}
4493+
4494+
**Tags**:
4495+
4496+
<mark>agent_restart</mark>
4497+
4498+
**FQCN**:
4499+
4500+
`inputs.ebpf.cpu_balancer`
4501+
4502+
4503+
**Default value**:
4504+
```yaml
4505+
inputs:
4506+
ebpf:
4507+
cpu_balancer:
4508+
- nic_name: ""
4509+
ring_size: 4096
4510+
nic_rx_cpus: []
4511+
xdp_cpus: []
4512+
mode: 0
4513+
```
4514+
4515+
**Schema**:
4516+
| Key | Value |
4517+
| ---- | ---------------------------- |
4518+
| Type | dict |
4519+
4520+
**Description**:
4521+
4522+
Through the combination of RSS and XDP technologies, network traffic can be distributed
4523+
across different CPU cores to achieve more efficient network packet processing. The entire
4524+
path from the network interface card (NIC) to final processing is as follows:
4525+
4526+
`[NIC] --(interrupt)--> [RSS receive CPU] --(XDP CPUMAP)--> [Softirq CPU1, CPU2 ...]`
4527+
4528+
When the network interface card (NIC) receives a network packet, it triggers an interrupt
4529+
signal. This is a hardware-level interrupt used to notify the system that a packet has arrived.
4530+
Then, XDP CPUMAP is used to distribute the packet to other CPU cores for further processing.
4531+
XDP CPUMAP is a mechanism that maps network packets from the receiving CPU and distributes them
4532+
to multiple processing CPUs for more efficient soft interrupt handling.
4533+
When the network interface receives encapsulated packets such as GRE, Double VLAN (QinQ), or VXLAN,
4534+
which cannot be evenly distributed across CPUs using the network card's RSS (Receive Side Scaling),
4535+
XDP CPUMAP can effectively alleviate this issue. We can use the `top` command to check the soft
4536+
interrupt usage of each CPU and the `rxtxcpu` command to view the data collection counts for each CPU,
4537+
helping to determine whether there is an imbalance in CPU data processing.
4538+
4539+
`rxtxcpu` command:
4540+
```
4541+
# rxtxcpu -d rx <nic-name>
4542+
110 packets captured on cpu0.
4543+
93 packets captured on cpu1.
4544+
112 packets captured on cpu2.
4545+
77 packets captured on cpu3.
4546+
853 packets captured on cpu4.
4547+
53 packets captured on cpu5.
4548+
...
4549+
```
4550+
You can view the statistics of received packets for each CPU. If the count for a specific CPU is
4551+
significantly higher than others, it indicates an imbalance in CPU processing.
4552+
4553+
Configuration Item:
4554+
- nic_name: Network interface name. default value is ""
4555+
- ring_size: Receive (RX) ring buffer size, default value is 4096
4556+
- nic_rx_cpus: The list of CPUs that handle the network data received by the network interface
4557+
card (NIC), triggered by a physical interrupt. Since the network interface card (NIC) contains
4558+
multiple physical cores, and each physical core has multiple logical cores, we select the
4559+
physical core on the NUMA node where the NIC is located. This can be done as follows:
4560+
4561+
```
4562+
# deepflow-ebpfctl nicinfo --interface=p1p1
4563+
Device: p1p1
4564+
Address: 0000:41:00.0
4565+
Driver: i40e
4566+
Rx-Channels: 64
4567+
Tx-Channels: 64
4568+
RX-Ring-Size: 512
4569+
TX-Ring-Size: 512
4570+
PROMISC: 0
4571+
NumaNode: 1
4572+
4573+
# deepflow-ebpfctl cpu_layout show
4574+
cores = [0, 1, 2, 3, 4, 5]
4575+
sockets = [0, 1]
4576+
#
4577+
Socket 0 Socket 1
4578+
------------ ------------
4579+
Core 0 [0, 12] [1, 13]
4580+
Core 1 [2, 14] [3, 15]
4581+
Core 2 [4, 16] [5, 17]
4582+
Core 3 [6, 18] [7, 19]
4583+
Core 4 [8, 20] [9, 21]
4584+
Core 5 [10, 22] [11, 23]
4585+
```
4586+
From the above information, we know that the NIC p1p1 is located on NUMA node 1. We need to
4587+
select cores from Socket 1, for example: selecting Core 0, Core 1. From Core 0, we can choose
4588+
logical cores 1 or 13, and from Core 1, we can choose logical cores 3 or 15. The configuration
4589+
can be set as [13,15].
4590+
Default value is []
4591+
- xdp_cpus: XDP CPU distribution list, ensuring that the distributed CPUs do not overlap with nic_rx_cpus.
4592+
Default value is []
4593+
- mode: CPU Balance Mode, default value is 0.
4594+
4595+
Example:
4596+
```yaml
4597+
inputs:
4598+
ebpf:
4599+
cpu_balancer:
4600+
- nic_name: eth0
4601+
ring_size: 4096
4602+
nic_rx_cpus: [1,3]
4603+
xdp_cpus: [5,7,9,11]
4604+
mode: 0
4605+
- nic_name: eth1
4606+
ring_size: 4096
4607+
nic_rx_cpus: [13,15]
4608+
xdp_cpus: [17,18,19,21]
4609+
mode: 0
4610+
```
4611+
44924612
## Resources {#inputs.resources}
44934613

44944614
### Push Interval {#inputs.resources.push_interval}

0 commit comments

Comments
 (0)