Remaining TODO for rc3:

• Debug the Perlmutter & Compy errors
• Rerun jobs on Perlmutter & Compy (if only tests are changed, reuse the Unified rc2 environment; otherwise, use a dev environment)
• Determine what expected results to test against. Because the zppy tests aren't run frequently on Perlmutter & Compy, they are going to be out of date. Options:
  • Examine the diffs, checking if it's likely all of them are due to changes over this development period. If so, update the local machine's expected results.
  • Copy/transfer the Chrysalis expected results to Perlmutter & Compy and test against those. This is probably the more robust approach, unless the plotting packages produce results that differ greatly depending between machines.

The second commit, 0c97da8, undoes the NCO call change of #717. This needed to be undone because the NCO call was hard-coded to Chrysalis, thus causing the issue noted at #746 (comment).

Note that #734 should make this more painless, but I'm wary of adding a new parameter at this point in Unified testing. That should be an early PR in the next development period though.

Copy/transfer the Chrysalis expected results to Perlmutter & Compy and test against those. This is probably the more robust approach, unless the plotting packages produce results that differ greatly depending between machines.

I think this would actually be infeasible.

# cd into the expected results directory on Chrysalis
cd /lcrc/group/e3sm/public_html/zppy_test_resources

# Let's count how many times `lcrc` appears
grep -R -l "lcrc" * | wc -l
# 9465

# And how many times a `ac.<username>` path appears
grep -R -l "ac\." * | wc -l
# 934

In theory, we could systematically change the file paths per machine, but that's outside the scope of the Unified testing period. Therefore, we'll continue with what we've been doing, i.e., option 1:

Examine the diffs, checking if it's likely all of them are due to changes over this development period. If so, update the local machine's expected results.