Panan Optimization #113

edoyango · 2025-06-18T01:36:59Z

This is picking up from @minghangli-uni's work for helping optimize @claireyung's panan runs in preparation for her 10-year run.

Live results (5-day run):

Build	SUs	comment
`pr75-1` best of 3	3035.88	This is Helen's build where slowness was first noted
`pr113-2` best of 1	2931.41	`pr113-2` has a few changes: ifx instead of ifort, `-{march,mtune}=sapphirerapids` and spack build changes. I was a little worried since sometimes `ifx` can be a lot slower than `ifort`.
`pr113-3` best of 1	2935.11	adds `-O3`. Not much difference. This is because `-O3` isn't registered in most of our builds as `-O2` is hardcoded and placed after spack user-supplied `fflags`.
`pr113-4` best of 1	2992.43	changes `-march=sapphirerapids` to `-march=cascadelake` for compatibility with cascadelake nodes. A bit of a slow down is not surprising but still disappointing.
`pr113-10` best of 2	3028.48	Adds `-flto -fuse-ld=lld` to everything except CICE6. Disappointing result.
`pr113-20` best of 1	2957.30	Adds `-qopt-prefetch` to everything, resulting in a small improvement.
`pr113-21` best of 1	2928.64	Makes sure `-O3` takes precedence. Got a small boost.
`pr113-25` best of 1	2957.30	Removes `-flto` to see if performance is better (since previous `-flto` result showed a slowdown). Does actually slow down, so would prefer to keep it.
`pr113-26` best of 1	2947.13	Uses `[email protected]` after manodeep's encouraging findings. Not much difference.
`pr113-27` best of 1	2887.04	Adds back `-flto` Which seemed to help a decent amount.

Current most preferred: pr113-27 for backwards compatibility.

🚀 The latest prerelease access-om3/pr113-3 at 4899d3f is here: #113 (comment) 🚀

🚀 The latest prerelease access-om3/pr113-4 at 89258cb is here: #113 (comment) 🚀

🚀 The latest prerelease access-om3/pr113-14 at 8f4870d is here: #113 (comment) 🚀

🚀 The latest prerelease access-om3/pr113-18 at 0377ee8 is here: #113 (comment) 🚀

🚀 The latest prerelease access-om3/pr113-20 at 0377ee8 is here: #113 (comment) 🚀

🚀 The latest prerelease access-om3/pr113-21 at f176d6b is here: #113 (comment) 🚀

🚀 The latest prerelease access-om3/pr113-22 at 6d8dab5 is here: #113 (comment) 🚀

🚀 The latest prerelease access-om3/pr113-27 at 11938bd is here: #113 (comment) 🚀

🚀 The latest prerelease access-om3/pr113-28 at 7793329 is here: #113 (comment) 🚀

github-actions · 2025-06-18T01:42:13Z

🚀 Attempted to deploy access-om3 Prerelease pr113-1 with commit a99ce22

🖥️ `Gadi` Deployment ❌

github-actions · 2025-06-18T05:46:58Z

🚀 Attempted to deploy access-om3 Prerelease pr113-2 with commit 172ff22

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-2 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-2

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-2 environment.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 2025.06.000
access-nri/spack-config version 2025.02.2

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2025-07-01T06:38:21Z

🚀 Attempted to deploy access-om3 Prerelease pr113-3 with commit 4899d3f

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-3 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-3

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-3 environment.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 2025.06.000
access-nri/spack-config version 2025.06.001

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2025-07-02T00:10:11Z

🚀 Attempted to deploy access-om3 Prerelease pr113-4 with commit 89258cb

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-4 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-4

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-4 environment.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 2025.06.000
access-nri/spack-config version 2025.06.001

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2025-07-02T03:37:13Z

🚀 Attempted to deploy access-om3 Prerelease pr113-5 with commit c083130

🖥️ `Gadi` Deployment ❌

github-actions · 2025-07-02T04:07:42Z

🚀 Attempted to deploy access-om3 Prerelease pr113-6 with commit bda5064

🖥️ `Gadi` Deployment ❌

github-actions · 2025-07-02T04:45:01Z

🚀 Attempted to deploy access-om3 Prerelease pr113-7 with commit 84186ca

🖥️ `Gadi` Deployment ❌

github-actions · 2025-07-02T05:48:30Z

🚀 Attempted to deploy access-om3 Prerelease pr113-8 with commit 9d1467c

🖥️ `Gadi` Deployment ❌

github-actions · 2025-07-02T06:48:39Z

🚀 Attempted to deploy access-om3 Prerelease pr113-9 with commit f768314

🖥️ `Gadi` Deployment ❌

github-actions · 2025-07-02T08:28:27Z

🚀 Attempted to deploy access-om3 Prerelease pr113-10 with commit 621ff7f

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-10 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-10

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-10 environment.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 2025.06.000
access-nri/spack-config version 2025.06.001

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2025-07-03T05:23:48Z

🚀 Attempted to deploy access-om3 Prerelease pr113-18 with commit 0377ee8

🖥️ `Gadi` Deployment ❌

edoyango · 2025-07-03T05:39:53Z

!redeploy

github-actions · 2025-07-03T11:41:11Z

🚀 Attempted to deploy access-om3 Prerelease pr113-19 with commit 0377ee8

🖥️ `Gadi` Deployment ❌

edoyango · 2025-07-03T22:55:55Z

!redeploy

github-actions · 2025-07-04T00:16:06Z

🚀 Attempted to deploy access-om3 Prerelease pr113-20 with commit 0377ee8

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-20 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-20

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-20 environment.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 2025.06.000
access-nri/spack-config version 2025.06.001

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2025-07-04T05:54:06Z

🚀 Attempted to deploy access-om3 Prerelease pr113-21 with commit f176d6b

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-21 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-21

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-21 environment.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 97c26382fab997698a344b9aac364b8a4e7411df
access-nri/spack-config version 2025.06.001

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2025-07-04T10:54:12Z

🚀 Attempted to deploy access-om3 Prerelease pr113-22 with commit 6d8dab5

🖥️ `Gadi` Deployment ❌

edoyango · 2025-07-04T11:12:52Z

!redeploy

github-actions · 2025-07-04T17:14:07Z

🚀 Attempted to deploy access-om3 Prerelease pr113-23 with commit 6d8dab5

🖥️ `Gadi` Deployment ❌

edoyango · 2025-07-04T17:50:28Z

!redeploy

github-actions · 2025-07-04T20:55:23Z

🚀 Attempted to deploy access-om3 Prerelease pr113-24 with commit 6d8dab5

🖥️ `Gadi` Deployment ❌

github-actions · 2025-07-06T02:26:30Z

🚀 Attempted to deploy access-om3 Prerelease pr113-25 with commit f8a504f

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-25 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-25

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-25 environment.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 97c26382fab997698a344b9aac364b8a4e7411df
access-nri/spack-config version 2025.07.001

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2025-07-06T23:12:47Z

🚀 Attempted to deploy access-om3 Prerelease pr113-26 with commit 724dc87

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-26 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-26

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-26 environment.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 97c26382fab997698a344b9aac364b8a4e7411df
access-nri/spack-config version 2025.07.001

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2025-07-07T01:02:45Z

🚀 Attempted to deploy access-om3 Prerelease pr113-27 with commit 11938bd

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-27 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-27

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-27 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 97c26382fab997698a344b9aac364b8a4e7411df
access-nri/spack-config version 2025.07.001

If the above was not what was expected, commit changes to config/versions.json in this PR.

github-actions · 2025-07-17T01:39:30Z

🚀 Attempted to deploy access-om3 Prerelease pr113-28 with commit 7793329

🖥️ `Gadi` Deployment ✔️

Usage Instructions

access-om3, defined in ./spack.yaml, will be deployed to Gadi as:

2025.05.001 as a Release (when merged).
pr113-28 as a Prerelease (during this PR).

This Prerelease is accessible on Gadi using:

module use /g/data/vk83/prerelease/modules
module load access-om3/pr113-28

When using the above modules, the binaries shall be on your $PATH.

For advanced users, this Prerelease is also accessible on Gadi via /g/data/vk83/prerelease/apps/spack/0.22/spack in the access-om3-pr113-28 environment.
Due to inode-saving measures, one will have to manually untar the environment metadata before environment activation with tar -xf .spack-env .spack-env.tar. It will require one to have write privileges.

Configuration Information

This Prerelease is deployed using:

access-nri/spack on branch 0.22
access-nri/spack-packages version 97c26382fab997698a344b9aac364b8a4e7411df
access-nri/spack-config version 2025.07.001

If the above was not what was expected, commit changes to config/versions.json in this PR.

Diff is 4899d3f...11938bd

+mom_symmetric

a99ce22

edoyango had a problem deploying to Gadi Prerelease June 18, 2025 01:37 — with GitHub Actions Failure

remove +mom_symmetric, +asymmetric_mem

172ff22

edoyango temporarily deployed to Gadi Prerelease June 18, 2025 04:43 — with GitHub Actions Inactive

Add O3 and ldflags

4899d3f

edoyango temporarily deployed to Gadi Prerelease July 1, 2025 05:41 — with GitHub Actions Inactive

-march=cascadelake

89258cb

edoyango temporarily deployed to Gadi Prerelease July 1, 2025 23:15 — with GitHub Actions Inactive

-flto

c083130

edoyango had a problem deploying to Gadi Prerelease July 2, 2025 02:22 — with GitHub Actions Failure

-flto only access3

bda5064

edoyango had a problem deploying to Gadi Prerelease July 2, 2025 04:04 — with GitHub Actions Failure

Fix missing "

84186ca

edoyango had a problem deploying to Gadi Prerelease July 2, 2025 04:09 — with GitHub Actions Failure

Update spack.yaml

9d1467c

edoyango had a problem deploying to Gadi Prerelease July 2, 2025 04:55 — with GitHub Actions Failure

add back -flto and add ldflags=-fuse-ld=lld

f768314

edoyango had a problem deploying to Gadi Prerelease July 2, 2025 05:50 — with GitHub Actions Failure

-flto fail at cice

621ff7f

edoyango temporarily deployed to Gadi Prerelease July 2, 2025 07:10 — with GitHub Actions Inactive

edoyango mentioned this pull request Jul 3, 2025

Panan Optimization using ifort #122

Closed

use use-default-flag-handler branch

f176d6b

edoyango temporarily deployed to Gadi Prerelease July 4, 2025 04:34 — with GitHub Actions Inactive

-march=sapphirerapids

6d8dab5

edoyango had a problem deploying to Gadi Prerelease July 4, 2025 10:45 — with GitHub Actions Failure

no -flto

f8a504f

edoyango temporarily deployed to Gadi Prerelease July 6, 2025 01:28 — with GitHub Actions Inactive

oneapi 2025.2.0

724dc87

edoyango temporarily deployed to Gadi Prerelease July 6, 2025 21:52 — with GitHub Actions Inactive

use flto with 2025.2.0

11938bd

edoyango temporarily deployed to Gadi Prerelease July 6, 2025 23:27 — with GitHub Actions Inactive

edoyango mentioned this pull request Jul 9, 2025

Improved config claireyung/access-om3-configs#1

Draft

Try ompi 4.1.5

7793329

edoyango temporarily deployed to Gadi Prerelease July 16, 2025 23:52 — with GitHub Actions Inactive

claireyung added a commit that referenced this pull request Aug 29, 2025

Manually add Ed's flags from PR of panan optimisation #113

ccf5862

Diff is 4899d3f...11938bd

Panan Optimization #113

Are you sure you want to change the base?

Panan Optimization #113

Uh oh!

Conversation

edoyango commented Jun 18, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jun 18, 2025

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Jun 18, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Jul 1, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Jul 2, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Jul 2, 2025

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Jul 2, 2025

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Jul 2, 2025

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Jul 2, 2025

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Jul 2, 2025

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Jul 2, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Jul 3, 2025

🖥️ Gadi Deployment ❌

Uh oh!

edoyango commented Jul 3, 2025

Uh oh!

github-actions bot commented Jul 3, 2025

🖥️ Gadi Deployment ❌

Uh oh!

edoyango commented Jul 3, 2025

Uh oh!

github-actions bot commented Jul 4, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Jul 4, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Jul 4, 2025

🖥️ Gadi Deployment ❌

Uh oh!

edoyango commented Jul 4, 2025

Uh oh!

github-actions bot commented Jul 4, 2025

🖥️ Gadi Deployment ❌

Uh oh!

edoyango commented Jul 4, 2025

Uh oh!

github-actions bot commented Jul 4, 2025

🖥️ Gadi Deployment ❌

Uh oh!

github-actions bot commented Jul 6, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Jul 6, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Jul 7, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

github-actions bot commented Jul 17, 2025

🖥️ Gadi Deployment ✔️

Uh oh!

Reviewers

edoyango commented Jun 18, 2025 •

edited by github-actions bot

Loading

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ❌

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️

🖥️ `Gadi` Deployment ✔️