Required valid_range in ExtData2G.yaml for collection needing extrapolation

[yuanjianz]

Your name

Yuanjian Zhang

Your affiliation

WashU

What happened? What did you expect to happen?

In the TransportTracer ExtData2G case, a confusing error message is thrown after #3272.

Issue 1

As the result of GEOS-ESM/MAPL#4645, after removing valid_range for CEDS_CO collection, a meaningless "nf90_open, Cannot find file" error is thrown. This is because the literal unparsed "%" is passed to nf90_open to find the file.

After fixing the bug, however, another issue comes to the surface.

Issue 2

For monthly_nointerp, an extrapolation method is provided. However, per ExtData2G documentation and code implementation, it is necessary to provide valid_range for collection needing extrapolation if it has %y %m %d in its path. So a valid_range is necessary for CEDS_CO collection.

Fix

Issue 1 is easy to fix as in GEOS-ESM/MAPL#4645. But issue 2 needs special consideration. We might need to manually set the valid_range for users downloading only a subset of the data or otherwise it will trigger the error in #3272.

What are the steps to reproduce the bug?

Run Transport Tracer ExtData2G test case

Please attach any relevant configuration and log files.

No response

What GEOS-Chem version were you using?

14.7.1

What environment were you running GEOS-Chem on?

Local cluster

What compiler and version were you using?

gcc 13.3.0

Will you be addressing this bug yourself?

Yes, but I will need some help

In what configuration were you running GEOS-Chem?

GCHP

What simulation were you running?

TransportTracers

As what resolution were you running GEOS-Chem?

c24

What meterology fields did you use?

MERRA-2

Additional information

@lizziel

[lizziel]

@yuanjianz, thanks for opening this. I'm actually running into the same thing while testing on NASA discover.

[lizziel]

I think the best solution may be to change the design of valid_range to do all of these things:

1. Yaml file will have full range of data in the GEOS-Chem data archive repository as valid_range, bypassing the problem interpolation requiring valid_range.
2. The model will crash only if data for the overlap period of valid_range and simulation period is missing. Currently the model crashes if any of the data in valid_range is missing, even if decades prior to configured simulation start time.
3. Users can reduce the valid_range if they wish to extrapolate beyond a specific period for whatever reason, e.g. an older met year.

Does this make sense. @yuanjianz and @mcdon1718-h, please poke holes in this before I bring it to Ben Auer.

[yuanjianz]

@lizziel, I agree with your design and I believe it is for the best accomodation for GCHP.

I let Claude to investigate the causes of the crash when any missing data is present in valid_range.

What Claude finds

The crash is probably during initialization, the program tries to find metadata of the file. It will try for MAX_TRIALS, which is 10, before crashes. The usable_time is overwritten by valid_range(1), if it is present. So the program will crash if any of the first 10 time slices in the valid_range is not present. I guess this is probably for cases with extrapolation when you need to get the metadata from a file not at current_time.

File: ExtDataAbstractFileHandler.F90:185–203 (find_any_file)

useable_time = current_time
if (allocated(this%valid_range)) then
   useable_time = this%valid_range(1)
end if
call fill_grads_template(trial_file, this%file_template, time=useable_time, _RC)
inquire(file=trim(trial_file),exist=file_found)
...
do i=1, MAX_TRIALS                              ! MAX_TRIALS = 10
   useable_time = useable_time + this%frequency
   call fill_grads_template(trial_file, this%file_template, time=useable_time, _RC)
   inquire(file=trim(trial_file),exist=file_found)
   if (file_found) exit
enddo
if (fail_on_missing) then
   _ASSERT(file_found, "Could not find any file to open to determine metadata ...")

Called from ExtDataGridCompNG.F90:883 in GetLevs at init, for every non-/dev/null collection, just to pick one real file to read variable metadata (units, vartype, levels).

The way forward

So if this is the culprit, the engineering problem here is the behaviour of initializating file metedata. Currently if valid_range is present, it searches for only 10 slices of time starting from the beginning of valid_range otherwise fails. Maybe we can let it search through all of the valid_range? Or is it intentionally doing so to avoid hanging?