Skip to content

add support for GPUs using OpenACC #475

@tukss

Description

@tukss

@orvedahl documented some relevant tests for running Rayleigh on GPUs using OpenACC in this repository.

Before starting the actual port we should enable support for the relevant compilers in the Rayleigh build infrastructure. For Nvidia GPUs that would be the PGI compiler nvfortran. As it lacks support for quad-precision floating-point numbers, which we use in Math_Layer/Legendre_Polynomials.F90, we need to find a way around this.

There are several options to fix this:

  1. Rewrite Legendre_Polynomials.F90 to avoid the use of quad precision numbers. It needs to be checked if the precision is needed after all.
  2. Implement quad-precision through an external library, such as GMP.
  3. Build the code using gfortran with its OpenACC support for Nvidia-PTX offloading. I have tested this and it works, but the downside here is that we will essentially always have to build our own compiler on each Nvidia GPU enabled cluster.
  4. Build only Legendre_Polynomials.F90 with a different compiler (e.g., gfortran). This is hard to do, because the .mod file format of nvfortran and gfortran is not compatible.

I am leaning towards option 1 and if that doesn't work using option 2.

After the build system works, we should implement @orvedahl's changes to the loops and also explore if we can make use of direct GPU-to-GPU MPI communication. That would hopefully allow us to keep the data on the GPU for the whole computation, except for I/O.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions