Skip to content

Home

Jump to bottom
Erik Kluzek edited this page Oct 10, 2025 · 10 revisions

Welcome to the CMIP7_inputdata_processing wiki!

Open process allowing commits and push to main branch

Members of the repository can make commits on the main branch and push directly to it.

Example, workflow:

# In your home directory (or anywhere you like)
cd
git clone https://github.com/NCAR/CMIP7_inputdata_processing
cd CMIP7_inputdata_processing

# Navigate to the relevant subdirectory, add your script and README.txt
# If you need to create subdirectories into the relevant subdirectory, feel free to do that. 
# for instance Ben would put his script(s) and README.txt to create the emissions in: CMIP7_inputdata_processing/code/anthropogenic-emissions 


# Save and commit your changes
git status
git add --all
git commit -m "Add script for [brief description]"
git push

Only one branch -- main

Only one branch is allowed the main branch

Recommendations for documentation files to add

Add a README.txt plain text file or README.md markdown file to the relevent directory and add information to it.

Here's a sample README.md to use: Sample README.md

Recommendations for scripts

  • Be sure to include information on where processing was done (assuming Derecho)
  • If specific environments need to be used (specific conda environments for example).
  • Prefer: python or Jupyter notebooks
  • Allow: Fortran, C, R, bash, etc.
  • Should avoid: NCL (if possible), perl
  • Should avoid proprietary languages requiring licenses: IDL, etc.
  • However, ALWAYS provide the scripts that were used in the language they were written in as they were to create the datasets

Recommendations for Testing of scripts

  • You should run through the documented steps for new scripts brought in
  • Update the documentation or the script if the process fails
  • Make sure everything needed to run them is documented (machine, modules loaded, conda environments used, etc.)
  • Test existing scripts similarly if you bring in changes to them
  • Add test scripts to make it easier to test that scripts work
  • Add automated tests that can be triggered to run when changes are made to scripts (github actions)

Can make Pull Requests (PR's) if desired

For larger changes, members may want to create their own fork of the repository. And then create a feature branch on their fork, that they open as a Pull Request in the repository. This is useful for getting feedback and code review from others before pushing your changes to the repository.

  • PR's are NOT required only optional.
  • Approvals are NOT required on PR's.
  • You can ask for specific people to review when a PR is opened.

Linking to other repositoriies

We have the repository setup with git-fleximod so that submodules pointing to other repositories can be done. The README_GITFLEXIMOD.rst at the top level gives instructions on how to use. But the short of it is to add links to other repositorys in the top levle .gitmodules file, and then run

./bin/git-fleximod update

Repositories linked to in this way, should be in either the NCAR or the ESCOMP orginizations on github, and NOT be personal forks. Personal forks are tied to a person, and may be lost or deleted if that person moves to a different position, wins the lottery or even just retires. Repositories should also point to a tag in the given repository, but a hash is also acceptable.

to bring in the update to those repositories.

Permissions and adding other members

Members have permissions to do most things in the repository. If a new member needs to be added -- ask @dattilo @rrbuchholz @cecilehanny or @ekluzek to add them.

Please add: issues, Discussions, or Projects

Members should feel free to add new issues to track their work, create or contribute to discussions, or to add projects if any of those are helpful for their work.

What to do when you find a problem with one of the scripts

If you find a bug in one of the scripts in the repository:

  1. Create an issue that documents the problem
  2. Figure out what the impact might be for simulations using the datasets created with it
  3. Let relevant people know about the problem and the impacts
  4. Figure out how to fix the problem
  5. Report what you find in the issue
  6. Test your fix out (see the notes on testing above)
  7. Push a fix for the problem to the repository
  8. Repeat the cycle again if a different problem is found

Naming convention for CMIP7_inputdata_processing tags

Tags will ONLY be made by someone on the admin team. This will be done on occasion at important points where the entire repository is completely setup or nearing that point.

Eventually we will need to make tags in this repository. At that point we will need a tagging convention. Since, these tags will be tied to a specific CESM release series, I suggest it should have that as part of the name. However, it's NOT tied to a specific CESM tag, so shouldn't have a specific cesm alpha or beta tag in the name. And probably should avoid looking too similar to a CESM tag.

Hence, maybe something like:

CMIP7_dataproc_cesm3_0_YYYYMMDD

Clone this wiki locally