[Bug-Candidate]: Poor scaling performance with high worker count - 80 workers 96 core cpu slower than 10 workers on a 12 core mac #1375
Description
Describe the issue:
Issue Summary
Echidna appears to show limited performance gains when increasing the number of workers on high-core-count systems. In our tests, running with 80 workers on AMD EPYC 96 core hardware resulted in slower execution times compared to 10 workers on a Mac M2, despite the server offering significantly more cores and comparable single-thread performance.
Performance Results
- Local (Mac M2, 10 workers): 72.35 seconds real time, user time 8m59s
- Remote (AMD EPYC, 80 workers): 104.12 seconds real time, , user time 90m18s
These results indicate that scaling is not linear, and may be affected by internal bottlenecks—potentially related to garbage collection, as noted in prior discussions with @gustavo-grieco
Configuration
- testLimit: 10,000,000
- Contract: Simple dummy test contract
- Identical Echidna configuration across both environments, differing only in worker count
Expected Behavior
With 8x more workers on similar single-core performance hardware, we anticipated faster execution on the AMD EPYC server, not 44% slower.
Actual Behavior
Instead, increasing the worker count on high core-count CPUs appears to negatively impact performance. This may point to parallelization inefficiencies or Haskell runtime limitations when scaling across many cores.
Environment
- Remote: AMD EPYC 9655 (96 cores) - 80 workers, 1.18 TB RAM
- Local: Apple M2 (12 cores) - 10 workers, 16gb RAM
- Echidna version: 2.2.6
Code example to reproduce the issue:
contract DummyTest {
uint256 public storedValue;
address public owner;
constructor() payable {
owner = msg.sender;
}
function setValues(uint256 _val, address _addr) public {
storedValue = _val;
owner = _addr;
}
}// Echidna relevant config
testLimit: 10000000
seqLen: 100
shrinkLimit: 1500
shrinkLimit: 1500
coverage: false
format: "text"Version:
2.2.6