Skip to content

index.md

Latest commit

 

History

History
196 lines (156 loc) · 7.82 KB

index.md

File metadata and controls

196 lines (156 loc) · 7.82 KB
Raw
layout default
title Home

ArgoDSM

ArgoDSM is a software distributed shared memory system.

Quickstart Guide

ArgoDSM uses the CMake build system. In the next section, we will describe how to build ArgoDSM from scratch. However, before doing that, some dependencies are required.

If you want to build the documentation, doxygen is required.

The only distributed backend ArgoDSM currently supports is the MPI one, so a compiler and libraries for MPI are required. If you are using OpenMPI, make sure you are running at least version 1.8.4. Installing OpenMPI is fairly easy, but you should contact your system administrator if you are uncertain about it.

ArgoDSM depends on the C++ QD Locking Library. This is included as a git submodule and is built automatically by CMake, requiring both git and an internet connection. For instructions on how to build offline, see building ArgoDSM offline.

Additionally, ArgoDSM requires libnuma to detect whether it is running on top of NUMA systems, and if so how they are structured internally.

Finally, C and C++ compilers that support the C17 and C++17 standards respectively are required.

Building ArgoDSM

Note: adjust the below commands to your needs, especially CMAKE_INSTALL_PREFIX.

You must activate exactly one of the ARGO_VM_* options. See [here]({{ site.baseurl }}/advanced.html#virtual-memory-management) for more details. ARGO_VM_MEMFD. If your kernel is too ancient and you cannot it, the next best option is ARGO_VM_SHM. However, your system administrator must mount /dev/shm with a tmpfs of sufficient size for your entire program (across all nodes). This is safe to do, as there will be no actual data stored in the location. If you cannot get enough space in /dev/shm, you must use ARGO_VM_ANONYMOUS, which in our experience is 20% to 40% slower.

git clone https://github.com/etascale/argodsm.git
cd argodsm && mkdir build && cd build
cmake -DARGO_BACKEND_MPI=ON              \
      -DARGO_BACKEND_SINGLENODE=ON       \
      -DARGO_TESTS=ON                    \
      -DARGO_VM_SHM=OFF                  \
      -DARGO_VM_ANONYMOUS=OFF            \
      -DARGO_VM_MEMFD=ON                 \
      -DBUILD_DOCUMENTATION=ON           \
      -DCMAKE_CXX_COMPILER=mpic++        \
      -DCMAKE_C_COMPILER=mpicc           \
      -DCMAKE_INSTALL_PREFIX=/usr/local/ \
      ../
make
make test
make install

Initially, you need to get the ArgoDSM sources. You can get the latest code from the Github repository.

git clone https://github.com/etascale/argodsm.git

CMake supports building in a separate folder. This is recommended for two reasons. First of all, it makes cleaning up much easier. Second, it allows for different builds with different configurations to exist in parallel.

cd argodsm && mkdir build && cd build

In order to generate the makefiles with CMake, you can use either the cmake or the ccmake tool. The difference is that the first one accepts all the build options as command line arguments, while the second one works interactively. Below is an example call to cmake with all the recommended command line arguments. If you plan on contributing to the ArgoDSM source code, you should also enable the ARGO_DEBUG option. Remember to change CMAKE_INSTALL_PREFIX to a path that you have write access to. After generating the makefiles proceed to building the library and executables with a simple make command.

If you are planning on building the ArgoDSM tests (recommended), CMake will automatically include the googletest framework as a submodule and build this as part of the project. To disable building googletest and the ArgoDSM test suite, set ARGO_TESTS=OFF.

cmake -DARGO_BACKEND_MPI=ON              \
      -DARGO_BACKEND_SINGLENODE=ON       \
      -DARGO_TESTS=ON                    \
      -DBUILD_DOCUMENTATION=ON           \
      -DCMAKE_CXX_COMPILER=mpic++        \
      -DCMAKE_C_COMPILER=mpicc           \
      -DCMAKE_INSTALL_PREFIX=/usr/local/ \
      ../
make

After the build process has finished, all the executables can be found in the bin directory. It will contain different subdirectories, one for each backend. It is highly recommended to run the test suite before proceeding, to ensure that everything is working fine.

If you want to build and manually compile your own applications, you can find the libraries in the lib directory. You need to link with the main libargo library, as well as exactly one of the backend libraries.

make test

This step executes the ArgoDSM tests. Tests for the MPI backend are run on two ArgoDSM software nodes by default. This number can be changed through setting the ARGO_TESTS_NPROCS CMake option to any number from 1 to 8. Keep in mind that some MPI distributions may not allow executing more processes than the number of physical cores in the system. Please note that some issues are only encountered when running with more than two ArgoDSM nodes or on multiple hardware nodes. For this reason it is highly recommended to run the MPI tests on at least four hardware nodes when possible. Refer to the next section to learn how to run applications on multiple hardware nodes.

make install

This step will copy the final ArgoDSM include files to /usr/local/include and libraries to /usr/local/lib/. You can choose a different path above if you want, but remember to either set LIBRARY_PATH, INCLUDE_PATH, and LD_LIBRARY_PATH accordingly, or provide the correct paths (-L, -I, and -Wl,-rpath,) to your compiler when compiling applications for ArgoDSM.

Running the Applications

For the singlenode backend, the executables can be run like any other normal executable.

For the MPI backend, they need to be run using the matching mpirun application. You should refer to your MPI's vendor documentation for more details. If you are using OpenMPI on a cluster with InfiniBand interconnects, we recommend the following:

mpirun --map-by ppr:1:node      \
       --mca mpi_leave_pinned 1 \
       --mca btl openib,self,sm \
       -n ${NNODES}             \
       ${EXECUTABLE}

${NNODES} is the number of hardware nodes you have available and ${EXECUTABLE} is the application to run. If you are not using InfiniBand, you can use the default OpenMPI transports by skipping the --mca btl openib,self,sm part of the command.

If you are running on a cluster, please consult your system administrator for details on how to access the cluster's resources. We recommend running ArgoDSM applications with one ArgoDSM node per hardware node. We also strongly recommend InfiniBand interconnects when using the MPI backend.

We strongly recommend running all of the MPI tests on at least two nodes before continueing working with ArgoDSM, to ensure that your system is configured properly and also that it is fully supported by ArgoDSM.

Building ArgoDSM offline

If you are building ArgoDSM in an offline environment, submodules will first have to be updated in an online environment. This can be done either by running CMake, manually executing git submodule update --init --recursive or using git clone --recurse-submodules https://github.com/etascale/argodsm.git to clone the ArgoDSM repository. Once submodules are updated, your ArgoDSM directory can safely be moved to the offline environment. Finally, before building offline, set GIT_SUBMODULE=OFF in CMake.

Contributing to ArgoDSM

If you are interested in contributing to ArgoDSM, do not hesitate to contact us. Please make sure that you understand the license terms.

Code Style

See Code Style

Code Review Process

See Code Style