nixlbench: failed to run multiple devices with backend GUSLI #1146
Description
nixlbench --backend=GUSLI
--device_list="0:K:/dev/nvme2n1,1:K:/dev/nvme2n2"
--num_initiator_dev=2
--num_target_dev=2
--op_type=READ
WARNING: Adjusting num_iter to 1008 to allow equal distribution to 1 threads
WARNING: Adjusting warmup_iter to 112 to allow equal distribution to 1 threads
Using null runtime for storage backend without ETCD
GUSLI backend: Automatically enabling storage_enable_direct for direct I/O
GUSLI backend initialized:
Client name: NIXLBench
Max simultaneous requests: 32
Direct I/O: Enabled (required)
Configured devices: 2
Device 0 [K]: /dev/nvme2n1 (sec=0x3)
Device 1 [K]: /dev/nvme2n2 (sec=0x3)
GUSLIc: initialized: 2 devices, log_fd=1 rv=0
Single instance storage backend - no synchronization needed
""
NIXLBench Configuration
""
Runtime (--runtime_type=[etcd]) : ETCD
ETCD Endpoint : disabled (storage backend)
Worker type (--worker_type=[nixl,nvshmem]) : nixl
Backend (--backend=[UCX,GDS,GDS_MT,POSIX,Mooncake,HF3FS,OBJ]): GUSLI
Enable pt (--enable_pt=[0,1]) : 0
Progress threads (--progress_threads=N) : 0
Device list (--device_list=dev1,dev2,...) : 0:K:/dev/nvme2n1,1:K:/dev/nvme2n2
Enable VMM (--enable_vmm=[0,1]) : 0
filepath (--filepath=path) :
Number of files (--num_files=N) : 1
Storage enable direct (--storage_enable_direct=[0,1]) : 1
Initiator seg type (--initiator_seg_type=[DRAM,VRAM]) : DRAM
Target seg type (--target_seg_type=[DRAM,VRAM]) : DRAM
Scheme (--scheme=[pairwise,manytoone,onetomany,tp]) : pairwise
Mode (--mode=[SG,MG]) : SG
Op type (--op_type=[READ,WRITE]) : READ
Check consistency (--check_consistency=[0,1]) : 0
Total buffer size (--total_buffer_size=N) : 8589934592
Num initiator dev (--num_initiator_dev=N) : 2
Num target dev (--num_target_dev=N) : 2
Start block size (--start_block_size=N) : 4096
Max block size (--max_block_size=N) : 67108864
Start batch size (--start_batch_size=N) : 1
Max batch size (--max_batch_size=N) : 1
Num iter (--num_iter=N) : 1008
Warmup iter (--warmup_iter=N) : 112
Large block iter factor (--large_blk_iter_ftr=N) : 16
Num threads (--num_threads=N) : 1
"----------------------------------------------------------------------------------------------------------------------------------------------------------------"
Block Size (B) Batch Size B/W (GB/Sec) Avg Lat. (us) Avg Prep (us) P99 Prep (us) Avg Post (us) P99 Post (us) Avg Tx (us) P99 Tx (us)
"----------------------------------------------------------------------------------------------------------------------------------------------------------------"
E1218 15:23:54.313923 84481 nixl_agent.cpp:938] createXferReq: no specified or potential backend had the required registrations to be able to do the transfer
createXferReq failed: NIXL_ERR_NOT_FOUND
GUSLIc: destroyed, pid=84481 rv=0