NastyMPI corrupts user-defined MPI data types

[devreal]

The delayed execution of put/get operations in combination with user-defined MPI data types leads to errors in otherwise correct programs. Consider the following example:

MPI_Win_create(&win);
MPI_Type_vector(..., &new_type);
MPI_Type_commit(&new_type);
MPI_Put(..., new_type, ...);
MPI_Type_destroy(&new_type); // perfectly legal
MPI_Win_flush(win); // <- the above put is deferred until here and executed with a type that has been destroyed

Reference: MPI standard 3.1, §4.1: MPI_Type_free

Marks the datatype object associated with datatype for deallocation and sets datatype
to MPI_DATATYPE_NULL . Any communication that is currently using this datatype will
complete normally.

As a consequence, NastyMPI has to keep track of type usage and destruction and (if necessary) defer type destruction until operations using that type have been started by NastyMPI.