Feat req: Refactor wrf model_mod and models/wrf directory, unify WRF/WRF-CHEM

[hkershaw-brown]

Use case

Refactor the wrf model_mod to make it easier/possible to:

• add state variables
• swap between wrf v3 and v4
• not assume the structure of the state variable in model_mod.

Is your feature request related to a problem?

The WRF model_mod has wrf_static_data_for_dart which is replicating functionality available in state_structure_mod.

• Adding variable X to the state means altering the model_mod code to have a wrf%dom(id)%type_X, rather than simply adding a line in the model_nml::wrf_state_variables.

• WRF has its own integer types (qtys), replicating (but not quite) the DART qty functionality.
  

  DART/models/wrf/model_mod.f90

  Lines 323 to 332 in 7ee9ed9

  	! JPH local variables to hold type indices
  	integer :: type_u, type_v, type_w, type_t, type_qv, type_qr, type_hdiab, &
  	type_qndrp, type_qnsnow, type_qnrain, type_qngraupel, type_qnice, &
  	type_qc, type_qg, type_qi, type_qs, type_gz, type_refl, type_fall_spd, &
  	type_dref, type_spdp, type_qh, type_qnhail, type_qhvol, type_qgvol, &
  	type_cldfra
  	integer :: type_u10, type_v10, type_t2, type_th2, type_q2, &
  	type_ps, type_mu, type_tsk, type_tslb, type_sh2o, &
  	type_smois, type_2dflash

  
  so you see code like this:
  

  DART/models/wrf/model_mod.f90

  Lines 889 to 890 in 7ee9ed9

  	var_type = wrf%dom(id)%var_type(var_id)
  	dart_type = wrf%dom(id)%dart_kind(var_id)

• WRF v3 vs. v4 - it is not clear for users what to change when moving from WRF v3 to v4. If possible, it would be good for the model_mod to be able to check the WRF version from the netcdf file.

• Wider issue Model_mods which use their own structure to keep track of state variables #389. Moving control of the state vector to core dart modules allows us to have general solutions for things like compression, removing missing values (so dart core code can assume every element of the state is valid), and other cool stuff.

Describe your preferred solution

Remove the state vector information from wrf_static_data_for_dart

Describe any alternatives you have considered

I think the wrf_static_data_for_dart is a road block for core dart development, however wrf-dart is widely used and has a lot of additional user developed code in circulation. There is 8000 lines of code in for wrf model_mod (+ unknown user modifications) so the refactoring needs to be approached with this in mind.
I think it is possible to simplify the bounds checks for each grid (staggered and various flavors of periodic), but I do not think the bounds check affects development of core dart.

[hkershaw-brown]

edit:
https://github.com/hkershaw-brown/DART/tree/wrf-refactor is the initial attempt, having a go at updating the wrf model_mod

https://github.com/NCAR/DART/tree/wrf-fresh is starting from scratch redoing the wrf model_mod

[hkershaw-brown]

wrf model_mod get_close has a test for missing_r8

https://github.com/NCAR/DART/blob/72e5740b71ef6c7cde4c6f946e2c1df2019365fb/models/wrf/model_mod.f90#L6419C3-L6428

should check for verti_is_undefined before doing this (or not have the missing_r8 check)

[hkershaw-brown]

Vertical Convertion

There are several references to the vertical location of state being converted in get_state_meta_data. This is incorrect, get_state_meta_data does not do vertical conversion. For example:

DART/models/wrf/model_mod.f90

Lines 6406 to 6417 in 72e5740

	! Convert local vertical coordinate to requested vertical coordinate if necessary.
	! This should only be necessary for obs priors, as state location information already
	! contains the correct vertical coordinate (filter_assim's call to get_state_meta_data).
	if (vertical_localization_on()) then
	if (local_which /= wrf%dom(1)%localization_coord) then
	call vert_convert(state_handle, local_loc, loc_qtys(t_ind), istatus2)
	! Store the "new" location into the original full local array
	locs(t_ind) = local_loc !HK Overwritting the location
	else
	istatus2 = 0
	endif
	endif

get_close_obs and get_close_state are identical - they both call get_close.
get_close calls vert_convert

convert_vertical_obs calls vert_convert
convert_vertical_state - builds columns from the state

• unneeded call to get horizontal location, the location is passed into convert_vertical_state
  

  DART/models/wrf/model_mod.f90

  Lines 3026 to 3027 in 70e6af8

  	! first obtain lat/lon from (ip,jp)
  	call get_wrf_horizontal_location( ip, jp, var_type, id, lon, lat )

• 🐛 Bug: convert_vertical_state is using the module localization coord, not the which_vert passed into convert_vertical_state.
  

  DART/models/wrf/model_mod.f90

  Line 3030 in 70e6af8

  if (wrf%dom(id)%localization_coord == VERTISLEVEL) then

• check all surface variables, model_pressure_distrib
  

  DART/models/wrf/model_mod.f90

  Lines 4807 to 4812 in 70e6af8

  	elseif( var_type == wrf%dom(id)%type_mu .or. var_type == wrf%dom(id)%type_tslb .or. &
  	var_type == wrf%dom(id)%type_ps .or. var_type == wrf%dom(id)%type_u10 .or. &
  	var_type == wrf%dom(id)%type_v10 .or. var_type == wrf%dom(id)%type_t2 .or. &
  	var_type == wrf%dom(id)%type_th2 .or. &
  	var_type == wrf%dom(id)%type_q2 .or. var_type == wrf%dom(id)%type_tsk .or. &
  	var_type == wrf%dom(id)%type_smois .or. var_type == wrf%dom(id)%type_sh2o) then

[hkershaw-brown]

To do:

• wrf has a namelist option for calendar type. Are there observations not in GREGORIAN (2d?)

• variable_is_on_domain - assuming <10 domains (i1):
  

  DART/models/wrf/model_mod.f90

  Lines 7745 to 7746 in f996f26

  	read(domain_id_string(i:i),'(i1)') domain_int
  	if ( domain_int == id ) variable_is_on_domain = .true.

• check whether map utils varies between WRF versions module_map_utils.F misc_definitions_module.F

• restrict_polar is hardcoded in two places (and is not a namelist option)
  

  DART/models/wrf/model_mod.f90

  Line 938 in 9729d78

  ! 10 = polar observation while restrict_polar namelist true

[hkershaw-brown]

For VERTISLEVEL we have:

DART/models/wrf/model_mod.f90

Lines 895 to 900 in 9729d78

	! return staggered levels for vertical types? check this later.
	if( (var_type == wrf%dom(id)%type_w ) .or. (var_type == wrf%dom(id)%type_gz) ) then
	lev = real(kp) - 0.5_r8
	else
	lev = real(kp)
	endif

bottom_top = 29 ;
bottom_top_stag = 30 ;
float W(Time, bottom_top_stag, south_north, west_east) ;

Edit: then this:

DART/models/wrf/model_mod.f90

Lines 1857 to 1858 in 9729d78

	! Adjust zloc for staggered ZNW grid (or W-grid, as compared to ZNU or M-grid)
	zloc = zloc + 0.5_r8

[hkershaw-brown]

note in the code about latlon project settings only valid for global domains

DART/models/wrf/model_mod.f90

Lines 7307 to 7316 in 9729d78

	!JPH --- this latinc/loninc calculations are only valid for global domains
	if ( wrf%dom(id)%scm ) then
	! JPH -- set to zero which should cause the map utils to return NaNs if called
	latinc = 0.0_r8
	loninc = 0.0_r8
	else
	latinc = 180.0_r8/wrf%dom(id)%sn
	loninc = 360.0_r8/wrf%dom(id)%we
	endif

[nancycollins]

for help with the acronyms:

• code marked 'nc' is actually code done by greg lawson with minimal input from me
• 'scm' is single column model and was done by josh hacker (JPH)
• the vortex forward operator code was done by ryan torn (and eventually should be pulled out into an obs_def module

comments on these particular few lines:
'global' doesn't seem to be the opposite of single column, but i'm also sure i don't understand exactly what projection settings are needed to use the wrf code that converts a location into an index into the wrf grid (which is how this model_mod works). most wrf cases are NOT global - they are regional grids. perhaps latinc and loninc aren't used in the regional case.

[hkershaw-brown]

Mask variables: XLAND, QTY_SURFACE_TYPE

Should you be interpolating a mask?
edit: rounded to the nearest integer in rttov obs_def:

DART/observations/forward_operators/obs_def_rttov13_mod.f90

Line 2065 in 9729d78

surftype = nint(atmos%surftype(imem))

wrf netcdf file XLAND:description = "LAND MASK (1 FOR LAND, 2 FOR WATER)" ;
comment and QTY_SURFACE_TYPE ! 0 = land, 1 = water, 2 = sea ice (not yet supported)

edit:
wrf 1 for land, rttov forward operator expecting 0 for land.
QTY_SURFACE_TYPE has the -1, QTY_LAND_MASK does not.

DART/models/wrf/model_mod.f90

Lines 2825 to 2843 in 9729d78

	else if( obs_kind == QTY_SURFACE_TYPE ) then ! 0 = land, 1 = water, 2 = sea ice (not yet supported)
	if ( debug ) print*,'Getting land mask'
	! Check to make sure retrieved integer gridpoints are in valid range
	if ( boundsCheck( i, wrf%dom(id)%periodic_x, id, dim=1, type=wrf%dom(id)%type_t ) .and. &
	boundsCheck( j, wrf%dom(id)%polar, id, dim=2, type=wrf%dom(id)%type_t ) ) then
	call getCorners(i, j, id, wrf%dom(id)%type_t, ll, ul, lr, ur, rc )
	if ( rc .ne. 0 ) &
	print*, 'model_mod.f90 :: model_interpolate :: getCorners XLAND rc = ', rc
	! Interpolation for the XLAND field -- XLAND is NOT part of state vector x, but rather
	! in the associated domain meta data
	fld(1, :) = dym*( dxm*real(wrf%dom(id)%land(ll(1), ll(2))) + &
	dx*real(wrf%dom(id)%land(lr(1), lr(2))) ) + &
	dy*( dxm*real(wrf%dom(id)%land(ul(1), ul(2))) + &
	dx*real(wrf%dom(id)%land(ur(1), ur(2))) ) - 1
	endif

DART/models/wrf/model_mod.f90

Lines 2801 to 2823 in 9729d78

	! Land Mask has been added to accommodate satellite observations.
	! XLAND is not in the dart_ind vector, so get it from wrf%dom(id)%land
	else if( obs_kind == QTY_LANDMASK ) then
	if( my_task_id() == 0 ) print*, '*** Land mask forward operator not tested'
	if ( debug ) print*,'Getting land mask'
	! Check to make sure retrieved integer gridpoints are in valid range
	if ( boundsCheck( i, wrf%dom(id)%periodic_x, id, dim=1, type=wrf%dom(id)%type_t ) .and. &
	boundsCheck( j, wrf%dom(id)%polar, id, dim=2, type=wrf%dom(id)%type_t ) ) then
	call getCorners(i, j, id, wrf%dom(id)%type_t, ll, ul, lr, ur, rc )
	if ( rc .ne. 0 ) &
	print*, 'model_mod.f90 :: model_interpolate :: getCorners XLAND rc = ', rc
	! Interpolation for the XLAND field -- XLAND is NOT part of state vector x, but rather
	! in the associated domain meta data
	fld(1, :) = dym*( dxm*real(wrf%dom(id)%land(ll(1), ll(2))) + &
	dx*real(wrf%dom(id)%land(lr(1), lr(2))) ) + &
	dy*( dxm*real(wrf%dom(id)%land(ul(1), ul(2))) + &
	dx*real(wrf%dom(id)%land(ur(1), ur(2))) )
	endif

[hkershaw-brown]

I'm not sure what is not allowed here

DART/models/wrf/model_mod.f90

Lines 6738 to 6740 in 9729d78

	! As of 22 Oct 2007, this option is not allowed!
	! Note that j = 0 can only happen if we are on the M (or U) wrt to latitude
	if ( wrf%dom(id)%polar .and. j == 0 .and. .not. restrict_polar ) then

There are some notes in the code about periodic_x,y and polar

DART/models/wrf/model_mod.f90

Line 7092 in 9729d78

subroutine assign_boundary_conditions(id)

It is looking like these are still wrf namelist options and not in the wrf output netcdf file:
namelist_variables
wrfoutput