Use MPI_Init_thread requesting the necessary thread support level

If MPI is used in a multi-threaded environment the MPI implementation has
to be aware of the calling context of MPI methods. Therefore, initialize
the MPI environment with MPI_Init_thread requesting the desired thread
support level. So far, ls1 built with OpenMP support makes MPI calls only
from the main thread, i.e., uses MPI_THREAD_FUNNELED. For details see the
MPI standard.

Signed-off-by: Christoph Niethammer <[email protected]>

src/MarDyn.cpp

@@ -132,8 +132,20 @@ int run_unit_tests(const Values &options, const std::vector<std::string> &args)
  */
 int main(int argc, char** argv) {
 #ifdef ENABLE_MPI
-	MPI_Init(&argc, &argv);
-#endif
+#if defined(_OPENMP)
+	const int thread_support_level_requested = MPI_THREAD_FUNNELED; /* So far only the main thread makes MPI calls */
+#else
+	const int thread_support_level_requested = MPI_THREAD_SINGLE; /* No thread support required */
+#endif /* _OPENMP */
+	int thread_support_level_provided;
+	MPI_Init_thread(&argc, &argv, thread_support_level_requested, &thread_support_level_provided);
+	if (thread_support_level_provided < thread_support_level_requested) {
+		/* The logging infrastructure is not initalized yet, therfore perform manual output and exit. */
+		std::cerr << "ERROR: ls1 was build with OpenMP. However, the MPI implementation does not provide the necessary thread support level." << std::endl;
+		MPI_Abort(MPI_COMM_WORLD, EXIT_FAILURE);
+		::exit(EXIT_FAILURE);
+	}
+#endif /* ENABLE_MPI */
 
 	/* Initialize the global log file */
 	Log::global_log = new Log::Logger(Log::Info);