Skip to content

WIP: add fused interface 3D GPU kernels#3077

Draft
MarcoArtiano wants to merge 1 commit into
ma/noncons_gpufrom
ma/fused_interface
Draft

WIP: add fused interface 3D GPU kernels#3077
MarcoArtiano wants to merge 1 commit into
ma/noncons_gpufrom
ma/fused_interface

Conversation

@MarcoArtiano

@MarcoArtiano MarcoArtiano commented Jun 11, 2026

Copy link
Copy Markdown
Contributor
  • Clean up from old kernels (now they are commented)
  • Decide if keeping also combined false with nonconservative terms (at the moment we do not use them or test them)

main

────────────────────────────────────────────────────────────────────────────────────────────────────
Trixi.jl simulation finished.  Final time: 1.0  Time steps: 327 (accepted), 327 (total)
────────────────────────────────────────────────────────────────────────────────────────────────────

─────────────────────────────────────────────────────────────────────────────────
           Trixi.jl                     Time                    Allocations
                               ───────────────────────   ────────────────────────
       Tot / % measured:            96.8s /  97.7%            454MiB /  84.5%

Section                ncalls     time    %tot     avg     alloc    %tot      avg
─────────────────────────────────────────────────────────────────────────────────
rhs!                    1.64k    92.6s   97.9%  56.6ms    381MiB   99.2%   238KiB
  volume integral       1.64k    35.5s   37.5%  21.7ms    120MiB   31.4%  75.3KiB
  prolong2interfaces    1.64k    24.4s   25.8%  14.9ms   11.9MiB    3.1%  7.43KiB
  interface flux        1.64k    13.8s   14.6%  8.45ms   68.1MiB   17.7%  42.6KiB
  reset ∂u/∂t           1.64k    7.01s    7.4%  4.28ms   12.3MiB    3.2%  7.72KiB
  surface integral      1.64k    6.06s    6.4%  3.71ms   14.2MiB    3.7%  8.91KiB
  Jacobian              1.64k    2.78s    2.9%  1.70ms   18.2MiB    4.7%  11.4KiB
  boundary flux         1.64k    1.82s    1.9%  1.11ms    120MiB   31.2%  74.9KiB
  prolong2boundaries    1.64k    1.14s    1.2%   694μs   15.8MiB    4.1%  9.90KiB
  ~rhs!~                1.64k   17.3ms    0.0%  10.6μs   9.33KiB    0.0%    5.84B
  prolong2mortars       1.64k   2.66ms    0.0%  1.62μs     0.00B    0.0%    0.00B
  source terms          1.64k   1.80ms    0.0%  1.10μs     0.00B    0.0%    0.00B
  mortar flux           1.64k   1.61ms    0.0%   986ns     0.00B    0.0%    0.00B
calculate dt              328    2.03s    2.1%  6.20ms   3.08MiB    0.8%  9.62KiB
─────────────────────────────────────────────────────────────────────────────────

this branch

────────────────────────────────────────────────────────────────────────────────────────────────────
Trixi.jl simulation finished.  Final time: 1.0  Time steps: 327 (accepted), 327 (total)
────────────────────────────────────────────────────────────────────────────────────────────────────

─────────────────────────────────────────────────────────────────────────────────
           Trixi.jl                     Time                    Allocations
                               ───────────────────────   ────────────────────────
       Tot / % measured:            64.3s /  97.2%            429MiB /  87.3%

Section                ncalls     time    %tot     avg     alloc    %tot      avg
─────────────────────────────────────────────────────────────────────────────────
rhs!                    1.64k    60.4s   96.7%  36.9ms    371MiB   99.2%   232KiB
  volume integral       1.64k    35.9s   57.5%  22.0ms    120MiB   32.2%  75.3KiB
  reset ∂u/∂t           1.64k    7.08s   11.3%  4.33ms   12.3MiB    3.3%  7.72KiB
  surface integral      1.64k    6.06s    9.7%  3.71ms   14.2MiB    3.8%  8.91KiB
  prolong2interfaces    1.64k    5.57s    8.9%  3.40ms   70.6MiB   18.9%  44.2KiB
  Jacobian              1.64k    2.84s    4.6%  1.74ms   18.2MiB    4.9%  11.4KiB
  boundary flux         1.64k    1.84s    2.9%  1.12ms    120MiB   32.0%  74.9KiB
  prolong2boundaries    1.64k    1.09s    1.7%   664μs   15.8MiB    4.2%  9.90KiB
  ~rhs!~                1.64k   21.0ms    0.0%  12.9μs   9.33KiB    0.0%    5.84B
  prolong2mortars       1.64k   3.55ms    0.0%  2.17μs     0.00B    0.0%    0.00B
  interface flux        1.64k   2.35ms    0.0%  1.44μs     0.00B    0.0%    0.00B
  source terms          1.64k   2.30ms    0.0%  1.40μs     0.00B    0.0%    0.00B
  mortar flux           1.64k   2.12ms    0.0%  1.30μs     0.00B    0.0%    0.00B
calculate dt              328    2.05s    3.3%  6.26ms   3.07MiB    0.8%  9.58KiB
─────────────────────────────────────────────────────────────────────────────────

As we can see from the summary callback, the reset function takes an unreasonably amount of time, and probably we should try to fuse jacobian and calc source terms. These are very easy to fuse, as they share the same structure. Instead fusing surface integral with the aforementioned kernels might be a little bit trickier and therefore might not be worth the effort.

We should probably go back to the volume kernel and introduce same type of local share memory (kind of following the same spirit as the turbo version on CPU) and providing also a structure for a turbo version where the primitive variables are precomputed. These steps follows also the optimizations done in the recent paper from Waterhouse and coauthors, about Entropy Stable Euler on GPU.

This PR was created with assistance from Claude.

@github-actions

Copy link
Copy Markdown
Contributor

Review checklist

This checklist is meant to assist creators of PRs (to let them know what reviewers will typically look for) and reviewers (to guide them in a structured review process). Items do not need to be checked explicitly for a PR to be eligible for merging.

Purpose and scope

  • The PR has a single goal that is clear from the PR title and/or description.
  • All code changes represent a single set of modifications that logically belong together.
  • No more than 500 lines of code are changed or there is no obvious way to split the PR into multiple PRs.

Code quality

  • The code can be understood easily.
  • Newly introduced names for variables etc. are self-descriptive and consistent with existing naming conventions.
  • There are no redundancies that can be removed by simple modularization/refactoring.
  • There are no leftover debug statements or commented code sections.
  • The code adheres to our conventions and style guide, and to the Julia guidelines.

Documentation

  • New functions and types are documented with a docstring or top-level comment.
  • Relevant publications are referenced in docstrings (see example for formatting).
  • Inline comments are used to document longer or unusual code sections.
  • Comments describe intent ("why?") and not just functionality ("what?").
  • If the PR introduces a significant change or new feature, it is documented in NEWS.md with its PR number.

Testing

  • The PR passes all tests.
  • New or modified lines of code are covered by tests.
  • New or modified tests run in less then 10 seconds.

Performance

  • There are no type instabilities or memory allocations in performance-critical parts.
  • If the PR intent is to improve performance, before/after time measurements are posted in the PR.

Verification

  • The correctness of the code was verified using appropriate tests.
  • If new equations/methods are added, a convergence test has been run and the results
    are posted in the PR.

Created with ❤️ by the Trixi.jl community.

@MarcoArtiano MarcoArtiano added performance We are greedy gpu labels Jun 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gpu performance We are greedy

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant