CDAT Migration FY24 ‐ General Guide

Overview

This is a general guide on how to refactor diagnostic sets. It will cover how to get started, how to refactor code (generally), and how to perform regression testing.

• GitHub Project Tracker
  • This project tracker is used to map out progress and milestones.
• Root Development Branch: cdat-migration-fy24
  • This branch stores all of the developmental work for this task. We will merge this branch progressively into main when sets are refactored and pass regression testing.

Getting Started

1. Checkout the CDAT migration development branch
   • git checkout cdat-migration-fy24
2. Create a branch stemming from cdat-migration-fy24
   • git checkout -b refactor/<ISSUE#>-<SET-NAME>
   • Example: git checkout -b refactor/658-lat-lon-set
3. Create the development conda environment for your branch
   • mamba env create -f conda-env/dev.yml -n e3sm_diags_dev_658
4. Install the local development version of e3sm_diags for your branch
   • python -m pip install .
   • This ensures supplementary files are installed (e.g., .cfg files).
   • WARNING, if you make any changes to supplementary files AND/OR run e3sm_diags via CLI, you must repeat this command for those changes to be reflected in your environment.
5. Setup a test script(s) for the set you are refactoring
   • Running a test script provides quicker feedback with your code.
   • For example, I used ex1.py for refactoring the lat_lon set.
   • Additionally, I combine test scripts with VS Code's debugging capabilities to step through the code for real-time feedback. This results in an even more efficient development experience, compared writing print/logger statements and waiting for a response.
6. Create a draft pull request early using cdat-migration-fy24 as the base branch

Refactoring a Diagnostic Set

There are three steps in this process: 1. Refactor CDAT logic, 2. Clean up and refactor some more, 3. Regression testing.

1. Refactor CDAT Logic

Objective: Refactor CDAT logic with Xarray/xCDAT and successfully produce metrics .json and .png files

1. Find the description of the diagnostic set you will be refactoring here.
   • The core components of a set consist of a driver, plotter, viewer, and some utilities.
   • We'll figure out how to refactor the viewer at a later time in #628.
2. (RECOMMENDED) Analyze the core components for any direct code or utility imports that use CDAT and make an outline
   • Example outline
3. (RECOMMENDED) Plan how you will refactor the CDAT portions
   • If possible, write failing unit tests beforehand to cover edge cases (test-driven development)
4. Start refactoring CDAT logic with Xarray/xCDAT
   • Refer to the lat_lon set (PR #677) for some guidance
   • Try to reuse as much code as possible. For example, general classes dataset_xr.Dataset and utility functions in metrics.py, io.py, regrid.py
5. Run your test scripts to get feedback
   • Read the stack trace, understand how new code is behaving, fix any issues
   • NOTE: Make sure to python -m pip install . if running via python <script_name>.py to get the latest code changes in your environment. You don't need to do this if you're running with VS Code's Python interactive console and debugger because imports from the local package directory will take precedence.
6. Repeat steps until the diagnostic set can produce metrics (.json) and plots (.png)

2. Clean up and refactor some more

Objective: Implement readable, maintainable, and testable code

1. Refactor sub-optimal or hard to understand code
   • Excessive for loops, repeated logic
2. Break up large functions into smaller, maintainable functions
3. Write/fix unit tests for refactored code, if possible
   • /tests/e3sm_diags stores unit tests .py files
   • pytest command runs unit tests and generated code coverage report (tests_coverage_report/)

ALTERNATIVE: Write a TODO: ... statement

• Get back to refactoring at a later time
• If you are not confident in rewriting cleaner code for an implementation, skip it for now.
• Additional refactoring can be risky because there is minimal unit test coverage (easy to unknowingly introduce incorrect behaviors, side-effects, etc.)

3. Regression testing

(This section is a work-in-progress)

Objective: Regression testing is performed to ensure that a diagnostic set's metrics produced by your branch's refactored code is reasonably close to main

You will be using the auxiliary_tools/issue-658-compare-metrics.py script, which compares the .json of the dev branch against main. Overview of this script:

1. Takes two paths for .json files, one for the dev branch and the other for main.
2. Produces an Excel sheet listing the absolute and relative differences for the metrics of each variable
   • Relative differences are more useful in most cases because it measures the SCALE of the difference in percentage term, rather than just a raw number.
   • Absolute diff threshold 10^-e5 (0.00001)
   • Relative diff threshold is 10^e-1 (0.01 -> 1%)
3. Pinpoint the highest differences (if any) and try debugging why this is happening
   • Are the metric functions producing the same outputs?
   • Are derived variables correct?

General Tips

Set up VS Code

1. Follow this guide for getting started with VS Code with the recommended extensions. Also grab Remote SSH to use VS Code with remote machines via SSH.
2. If you are using VS Code, open the e3sm_diags.code-workspace file which automatically configures VS Code for you.
3. Create your mamba development environment (should be done already if you followed "Getting Started")
4. Configure VS Code with your mamba environment
   • https://code.visualstudio.com/docs/python/environments#_working-with-python-interpreters

Efficient Development and Debugging with VS Code

VS Code offers the ability to debug by stepping through the code stack in real-time. By installing the Python extension, you will automatically have access to this feature. I use this feature extensively to develop and debug e3sm_diags using test scripts.