|
4 | 4 | --- |
5 | 5 | # Understanding the Configuration File |
6 | 6 |
|
| 7 | +## Choosing an example config |
| 8 | + |
| 9 | +Read down the questions below and stop at the first one that matches what you're trying to do. Each section names the YAML, the binary that consumes it, and the build flags or hardware it requires. **Backend selection is a build-time choice via `DAQIRI_MGR`** — the default build enables all three backends (DPDK raw, kernel sockets, and RDMA). |
| 10 | + |
| 11 | +??? question "1. I want to measure baseline throughput" |
| 12 | + Pick the backend that matches your stack, then the hardware or protocol variant. |
| 13 | + |
| 14 | + **DPDK raw** — runs on `daqiri_bench_raw_gpudirect`. Highest performance, kernel bypass; requires a Mellanox-class NIC. |
| 15 | + |
| 16 | + - **Generic discrete GPU** (template — replace `<placeholders>`) — `daqiri_bench_raw_tx_rx.yaml`. This is the file annotated line-by-line in the [walkthrough below](#annotated-walkthrough). |
| 17 | + - **DGX Spark / GB10** (prefilled) — `daqiri_bench_raw_tx_rx_spark.yaml`. `kind: host_pinned` for the integrated GPU; cores, PCIe addresses, and IPs are prefilled. See the [Spark profile callout](benchmarking_examples.md#update-the-loopback-configuration) for run details. |
| 18 | + - **No physical NIC available** — `daqiri_bench_raw_sw_loopback.yaml`. `loopback: "sw"`, no NIC required. Useful for first-time build verification, not representative of production performance. |
| 19 | + |
| 20 | + **RDMA / RoCE** — runs on `daqiri_bench_rdma` (use `--mode {tx,rx,both}`). Low-latency interconnect; available in the default build (set `-DDAQIRI_MGR="dpdk socket rdma"` explicitly for clarity). Requires an RDMA-capable fabric. Configs use `kind: host_pinned` regardless of platform. |
| 21 | + |
| 22 | + - **Generic** (template — replace IPs) — `daqiri_bench_rdma_tx_rx.yaml`. |
| 23 | + - **DGX Spark** (prefilled) — `daqiri_bench_rdma_tx_rx_spark.yaml`. See the [Spark profile callout](benchmarking_examples.md#update-the-loopback-configuration) for run details. |
| 24 | + |
| 25 | + **Kernel TCP/UDP sockets** — runs on `daqiri_bench_socket`. No NIC, no privileges, no special CMake flags. Useful as a comparison baseline against DPDK and RDMA. Both bind to `127.0.0.1`. |
| 26 | + |
| 27 | + - **UDP** — `daqiri_bench_socket_udp_tx_rx.yaml`. |
| 28 | + - **TCP** — `daqiri_bench_socket_tcp_tx_rx.yaml`. |
| 29 | + |
| 30 | +??? question "2. I have out-of-order UDP packets that need to be reordered on the GPU" |
| 31 | + DAQIRI's flagship pipeline: a CUDA kernel reads a sequence number from each packet's header and places packets at the correct offset in a GPU buffer, so a downstream consumer sees a fully ordered stream without a CPU touch. Configs run on `daqiri_bench_raw_reorder_seq` unless 2.4 applies. Sub-questions: |
| 32 | + |
| 33 | + **2.1 Which algorithm matches how your packets encode batches?** |
| 34 | + |
| 35 | + - *"My protocol sends a fixed N packets per logical batch; the seqno identifies position within the batch"* — `seq_packets_per_batch`. |
| 36 | + - *"My protocol identifies the batch index in the seqno; packets-per-batch is fixed at the protocol level"* — `seq_batch_number`. |
| 37 | + |
| 38 | + **2.2 Where should the reorder run?** |
| 39 | + |
| 40 | + - GPU kernel (default, recommended) — `reorder_type: "gpu"`. |
| 41 | + - CPU (throughput-bounded; comparison/baseline path) — `reorder_type: "cpu"`. |
| 42 | + |
| 43 | + **2.3 Self-contained, or do you have a TX peer?** |
| 44 | + |
| 45 | + - TX+RX — closed-loop in one process. |
| 46 | + - RX-only — you'll generate traffic separately. **A standalone run of any `raw_rx_*` config exits cleanly with `0` packets if no traffic arrives — that is not a bug; you need a TX peer.** |
| 47 | + |
| 48 | + **2.4 Do you also need an in-kernel payload type conversion?** |
| 49 | + |
| 50 | + - No — pick a leaf from the table below. |
| 51 | + - Yes — `daqiri_bench_raw_tx_rx_reorder_quantize_seq_batch.yaml` (runs on `daqiri_bench_raw_reorder_quantize`, not `daqiri_bench_raw_reorder_seq`). Combines `seq_batch_number` reorder with an in-kernel payload type conversion; the `data_types` block sets the input and output types (the example uses int4 → fp32). Pick this when wire format and compute format differ. |
| 52 | + |
| 53 | + Concrete leaves (without conversion): |
| 54 | + |
| 55 | + | YAML | Algorithm | Kernel | Direction | |
| 56 | + |---|---|---|---| |
| 57 | + | `daqiri_bench_raw_tx_rx_reorder_seq_1024.yaml` | `seq_packets_per_batch` (1024) | GPU | TX+RX | |
| 58 | + | `daqiri_bench_raw_tx_rx_reorder_seq_1024_cpu.yaml` | `seq_packets_per_batch` (1024) | CPU | TX+RX | |
| 59 | + | `daqiri_bench_raw_rx_reorder_seq_ppb.yaml` | `seq_packets_per_batch` (128) | GPU | RX-only | |
| 60 | + | `daqiri_bench_raw_rx_reorder_seq_batch.yaml` | `seq_batch_number` | GPU | RX-only | |
| 61 | + | `daqiri_bench_raw_sw_loopback_reorder_seq_1024.yaml` | `seq_packets_per_batch` (1024) | CPU | TX+RX, no NIC | |
| 62 | + |
| 63 | + *Requires: DPDK build + Mellanox-class NIC (or the SW-loopback variant for first-time validation).* |
| 64 | + |
| 65 | +??? question "3. I need to parse small per-packet metadata on the CPU while keeping payload on the GPU" |
| 66 | + - `daqiri_bench_raw_tx_rx_hds.yaml` (runs on `daqiri_bench_raw_hds`). |
| 67 | + |
| 68 | + Header-data split: segment 0 (CPU) holds the header, segment 1 (GPU) holds the payload via GPUDirect zero-copy. Pick this when the CPU needs to read small per-packet fields without ever touching the payload. |
| 69 | + |
| 70 | + *Requires: DPDK build + Mellanox-class NIC.* |
| 71 | + |
| 72 | +??? question "4. I need flow-based load balancing across multiple RX queues" |
| 73 | + - `daqiri_bench_raw_rx_multi_q.yaml` (runs on `daqiri_bench_raw_gpudirect`). |
| 74 | + |
| 75 | + RX-only by design — drive traffic from a separate peer. Demonstrates flow-rule-based routing across multiple RX queues, each pinned to its own CPU core. |
| 76 | + |
| 77 | + *Requires: DPDK build + Mellanox-class NIC + a separate TX traffic source.* |
| 78 | + |
| 79 | +??? question "5. I need to record packet data to disk" |
| 80 | + Sub-question: **which output format?** |
| 81 | + |
| 82 | + **5.1 Wireshark- / tcpdump-compatible PCAP** — runs on `daqiri_example_pcap_writer`. Default; works on any filesystem. Run shape: `daqiri_example_pcap_writer <yaml> <output.pcap> [--tx]` (omit `--tx` for an RX-only tcpdump-style capture). |
| 83 | + |
| 84 | + - **Hardware loopback** — `daqiri_example_pcap_writer_tx_rx.yaml`. |
| 85 | + - **No physical NIC available** — `daqiri_example_pcap_writer_sw_loopback.yaml`. |
| 86 | + |
| 87 | + *Requires: DPDK build. No special CMake flag.* |
| 88 | + |
| 89 | + **5.2 Zero-copy GPU → NVMe writes** (advanced) — runs on `daqiri_example_gds_write`. Pick this *only* if the GPU-to-disk zero-copy path is the specific subject of investigation; otherwise pick PCAP (5.1). |
| 90 | + |
| 91 | + - **Hardware loopback** — `daqiri_example_gds_write_tx_rx.yaml`. |
| 92 | + - **No physical NIC available** — `daqiri_example_gds_write_sw_loopback.yaml`. |
| 93 | + |
| 94 | + *Requires: built with `-DDAQIRI_ENABLE_GDS=ON`, NVMe-backed storage, working cuFile / `nvidia_fs` stack, `gdscheck.py -p` reports `NVMe : Supported`.* |
| 95 | + |
| 96 | +## Annotated walkthrough |
| 97 | + |
7 | 98 | This section walks through the YAML configuration used by the benchmark applications. The annotated example below is based on `daqiri_bench_raw_tx_rx.yaml`. Click on the :material-plus-circle: icons to expand explanations for each annotated line. |
8 | 99 |
|
9 | 100 | Annotations are prefixed with a category: |
|
0 commit comments