-
Notifications
You must be signed in to change notification settings - Fork 154
GASNet: Known Issues and Workarounds
The following error may occur when too many application threads attempt to access GASNet:
GASNet Extended API: Too many simultaneous local client threads (limit=256). To raise this limit, configure GASNet using --with-max-pthreads-per-node=N
The workaround is to set -gex:immediate 0.
See #1898 for details.
GASNet applications can hang when running with a large number of GPUs per node. This occurs because Realm's GASNet layer allocates a separate output buffer for each pair of GPUs that communicate.
This can be worked around by calculating the worst-case number of required output buffers with the following formula:
(4 + 2 * gpus/node) * nodes
Then pass this number to -gex:obcount.
See realm#239 for details.
A hang with GASNet-EX has been observed to occur, particularly on Slingshot-11 based systems. Currently this is believed to be associated with Realm's GASNet-EX implementation, though debugging is still ongoing.
See #1857 for details.