[Question] For intra-node communication, shm is only used and many rc_verbs qps are created.

[redbrick9]

Hi there,

Here is my setup env configuration.
UCX_TLS=sm,tcp.rc_verbs

With the config, I noticed there are n * n qp connections for intra-node communications, even though the communication for intra-node only used shm. I'm concerned, if n is larger, a lot of resource are consumed in this way, even though these resources were not used for real communition.

I would like to have some configuration that shm is only used for intra-node communication and don't init rc
verbs qp. rc_verbs can be used for inter-node comm.

Thanks,
redbrick9

[tvegas1]

I do not see an easy way. You could probably use two UCP contexts, one with the configuration UCX_TLS=^ib for intra-node, and the other for inter-node. You would create intra-node UCP endpoints on intra-node context. But then you would get two workers to progress, and more memory consumption.