Skip to content

[Bug]: apsi_test测试时,若接收方的数据量达到某一阈值程序会卡在某处,不执行也不终止 #148

@WandQ

Description

@WandQ

Describe the bug

首次在本地16G的mac上,执行项目中的psi/apsi/apsi_test.cc代码无误,但当我将TestParams{100, 100000, false}修改为TestParams{10000, 100000, false}后,程序会卡住,日志如下:

[2024-07-12 07:46:20.569] [info] [ecdh_oprf_selector.cc:33] use fourq
[2024-07-12 07:46:20.697] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:20.697] [info] [thread_pool.cc:30] Create a fixed thread pool with size 7
[2024-07-12 07:46:20.719] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:20.721] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:20.727] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:20.728] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:20.729] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:20.750] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:20.753] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:20.758] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:20.758] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:20.759] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:20.781] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:20.784] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:20.788] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:20.789] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:20.790] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:20.811] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:20.814] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:20.818] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:20.819] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:20.820] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:20.842] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:20.845] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:20.849] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:20.849] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:20.850] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:20.874] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:20.877] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:20.882] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:20.883] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:20.884] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:20.907] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:20.910] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:20.915] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:20.915] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:20.916] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:20.942] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:20.945] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:20.947] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:20.948] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:20.949] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:20.971] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:20.974] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:20.978] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:20.978] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:20.979] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.001] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.004] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.006] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.007] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.008] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.029] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.032] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.034] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.034] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.036] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.055] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.058] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.060] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.061] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.062] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.117] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.119] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.121] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.122] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.123] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.144] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.147] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.149] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.149] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.151] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.172] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.174] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.178] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.178] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.179] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.199] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.202] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.205] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.205] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.207] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.230] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.233] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.236] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.237] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.238] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.263] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.266] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.268] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.269] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.271] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.296] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.299] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.302] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.302] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.304] [info] [sender_memdb.cc:696] Start inserting 5000 items in SenderMemDB
[2024-07-12 07:46:21.331] [info] [sender_memdb.cc:730] Found 5000 new items to insert in SenderDB
[2024-07-12 07:46:21.334] [info] [sender_memdb.cc:335] Launching 8 insert-or-assign worker tasks
[2024-07-12 07:46:21.337] [info] [sender_memdb.cc:351] Finished insert-or-assign worker tasks
[2024-07-12 07:46:21.337] [info] [sender_memdb.cc:748] Finished inserting 5000 items in SenderDB
[2024-07-12 07:46:21.338] [info] [sender_memdb.cc:945] count: 20, item_count:100000
[2024-07-12 07:46:21.338] [info] [sender_memdb.cc:526] Start generating bin bundle caches

到上面最后一条日志就卡住了,此时本机内存情况也是正常的。

然后我更换了一台内存更大的电脑,TestParams{10000, 100000, false}能够顺利执行,但TestParams{100000, 100000, false}仍然会出现上述问题。此外,卡住之后不会大量占用内存和CPU。

Steps To Reproduce

运行命令 bazel run -c opt //psi/apsi:apsi_test

Expected behavior

期望修复该问题,接收方10000的查询数据,应该也不是很大。

Version

v0.4.0.dev240524

Operating system

macbook m1 16G

Hardware Resources

8c16g

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions