[Bug]: Error reported by excluding `U` for `glb` atmsphere variable

[chengzhuzhang]

What happened?

This error is reported by Mark Z.: My config file contains a list of vars that I want create global average. I can confirm these are the requested ones in my settings file (ts_atm_monthly_glb_1979-2022-0044.settings). But in the output file, it is hitting an error because it is trying to regrid a variable that I didn’t ask for (which does not exist). Namely, I am trying to regrid OMEGA500, but for some reason the ts_atm_monthly_glb_1979-2022-0044.o44894299 file shows that I am asking for OMEGA50010.

More specifically, it seems like it took my variable request, which includes
...T,OMEGA500,U,U10,V,ICEFRAC…
And sort of concatenated / dropped out the ",U,U" portion of it to make:
...T,OMEGA50010,V,ICEFRAC...

I think this problem is caused by the following lines:

zppy/zppy/templates/ts.bash

Lines 48 to 56 in e4e1ed2

	{% if mapping_file == 'glb' -%}
	vars={{ vars }}
	# https://unix.stackexchange.com/questions/237297/the-fastest-way-to-remove-a-string-in-a-variable
	# https://stackoverflow.com/questions/26457052/remove-a-substring-from-a-bash-variable
	# Remove U, since it is a 3D variable and thus will not work with rgn_avg
	vars=${vars//,U}
	{%- else %}
	vars={{ vars }}
	{%- endif %}

I think at some point we can consider removing these lines, or make it more precisely to excluding 3d vars. note that V is also a 3d variable provided by the users. With the new NCO update from here #704, the user can generate global time-series for all 2d vars when var = "", (NCO takes care of fitering out multi-level fields).

What machine were you running on?

Perlmutter

Environment

zppy v3.0.0

What command did you run?

N/A

Copy your cfg file

N/A

What jobs are failing?

What stack trace are you encountering?

[forsyth2]

With the new NCO update from here #704, the user can generate global time-series for all 2d vars when var = "", (NCO takes care of [filtering] out multi-level fields).

That is, as you've noted, with #717 (the fix for #704), setting var ="" is equivalent to setting all vars except the 3D ones.

sort of concatenated / dropped out the ",U,U" portion

I think at some point we can consider removing these lines, or make it more precisely to excluding 3d vars.

We'd need a thorough list of 3D vars to exclude. But yes, I see if we had something like "remove U, U10, V", then the variable list fed to ncclimo would have been correctly formed.

So, I suppose the available options are:

1. Change vars=${vars//,U} to shopt -s extglob; vars=${vars//,@(U|U10|V)}, vars=${vars//,U@(10|)}; vars=${vars//,V}, or vars=$(echo "$vars" | sed 's/,U10\|,U\|,V//g') (and expanding that list to include any additional 3D variables).
2. Remove vars=${vars//,U} and have ncclimo fail when it hits the explicitly requested 3D variable.
3. Have Charlie update NCO to ignore explicitly requesting a 3D variable. Here, Charlie says:

If there is interest, I can make ncclimo ignore bad user choices (i.e., multi-level fields), though that can be a slippery slope