-
Notifications
You must be signed in to change notification settings - Fork 47
Open
Description
The DART Locality interface does not work if OpenMPI is started with the mpirun
default setting. This manifests in an erroneous locality hierarchy information, e.g. an invalid number of NUMA domains. On a local development system (projekt03) I can see 4 NUMA domains while there actually exist only 2. For whatever reason this depends on the process mapping. The issue currently causes strange behavior in DART.
In order to fix that we have to start with mpirun --map-by core -n [NPROCS] ./app
. This yields a correct locality information in DART. Maybe other options work as well, however, not with the default which is socket
.
I have checked this with OpenMPI versions 1.10.x and 2.0.x. Intel MPI, MVAPICH and MPICH work fine.