Allow Views of rank > 1 in collective communication operations

[nicoleavans]

The semantics used by Tpetra within Trilinos conflict with the limitation that SendView::rank > 1 not supported / RecvView::rank > 1 not supported. I have removed the two within src/KokkosComm/mpi/allreduce.hpp for my own experimentation replacing MPI with Kokkos Comm in Tpetra. I have also removed the equivalent lines from:

• src/KokkosComm/mpi/allgather.hpp
• src/KokkosComm/mpi/alltoall.hpp

This same style has proliferated through the NCCL communication backend (e.g., the corresponding NCCL allgather). A consensus on the style going forward should be reached. I support removing this limitation.

[dssgabriel]

As mentioned during today's meeting, I am in favor of lifting the multi-dimensional constraints on collective operations 🙂
This can be done for both the MPI and NCCL backends; however, we need well-defined semantics for multi-dimensional views, which should be reflected in our documentation.

This will be a very welcome change that broadens the capabilities of KokkosComm!

[nicoleavans]

Still need to update docs