Integration Test as a GitHub Action

[danineamati]

Currently, the integration test (specifically with HRRR: https://github.com/NCAR/miles-credit/blob/cb754d034afa61987cada23cdb6405ecf5078a9a/config/gen_2/smoke/integration_test_hrrr.yml) is configured as a "run as you see fit" configuration. To incorporate integration testing into the continuous integration process, we would benefit from:

1. A new GitHub action (not unlike https://github.com/NCAR/miles-credit/blob/cb754d034afa61987cada23cdb6405ecf5078a9a/.github/workflows/ci.yml). The open question is when it should run. On every pull request commit? When merged?
2. The save location needs to not depend on NCAR HPC:

   miles-credit/config/gen_2/smoke/integration_test_hrrr.yml

   Line 15 in cb754d0

   save_loc: '/glade/derecho/scratch/$USER/CREDIT_runs/gen2_integration_test_hrrr'

3. PBS should likely be removed:

   miles-credit/config/gen_2/smoke/integration_test_hrrr.yml

   Lines 240 to 249 in cb754d0

   	pbs: #derecho
   	conda: "credit-derecho"
   	project: "NAML0001"
   	job_name: "credit_integration_test_hrrr_gen2"
   	walltime: "01:00:00"
   	nodes: 8
   	ncpus: 64
   	ngpus: 4
   	mem: '480GB'
   	queue: 'main'

[djgagne]

When it should run depends on how long the test takes to complete and what the test covers. It is possible to trigger GH Actions workflows with a whole suite of options, including labels. @sgpearse has built the first CREDIT GH action workflow that runs on CIRRUS. Would the intent for this HRRR action be to run on a Github VM or on CIRRUS? The big advantages of CIRRUS are GPU access and the ability to read files from GLADE.