Continue SSP scenarios in ECHAM / AWI-ESM beyond 2100

[jannitzbon]

Hi all,

I would like to conduct "stabilization runs" with AWI-ESM-2.6 in which I would like to continue the simulation after an SSP scenario beyond 2100 for several decades. By default, all forcings and boundary conditions should be kept at the 2100 values and the model is just running freely into a new dynamic equilibrium state.

The desired way of doing this, would be, that it is possible to combine echam.scenario: sspXXX with a general.final_date > 2100-xx-xx. When I naively tried this, I ran into an error with the set_time_weight function already in year 2100 which is mentioned in this issue 567. However, I assume that even if 2100 would work, I would not be able to just continue beyond 2100 without any additional config changes.

I would appreciate any advice on how to approach this. I could imagine that @christian-stepanek @pgierz or @chrisdane might have previous experiences with this?

Thanks & Cheers,
Jan

[christian-stepanek]

I could imagine three ways how this could work. They would come with different levels of complexity of implementation:

1. (most simple on the technical side): run the model with SSP scenario forcing until 2100; then set up a new simulation with constant 2100 settings for all relevant forcings, using the restarts of the year 2100 of the SSP scenario as the initial state for all simulations; this necessitates two simulations, but should not come with major issues regarding the date if the time coordinate is kept continuous.
2. solve the issue that you describe (unknown difficulty)
3. (necessitates know-how by people like @pgierz who have implemented the scenario functionality into our ECHAM6-esm-tools framework): implement a new scenario setup in esm-tools where the relevant forcings are a combination of SSP until 2100 and the constant forcing thereafter. On the pool there is a forcing file that could be used as a template (it is for rcp45, though, so all values would have to be updated): /pool/data/ECHAM6/greenhouse.nc

If the necessity to run this kind of simulation would not likely occur regularly, I probably would stick to solution 1. It is a bit ugly, but how to run a simulation with fixed greenhouse forcing is a "three-liner" in namelist.echam. Or did I overlook the need for other modifications beyond the constant setting of greenhouse gases?

[jannitzbon]

Hi @christian-stepanek ,
thank you so much for these detailed suggestions and apologies for my delayed reply!

1. This indeed sounds like an easy solution, and it would be no problem to branch off a new simulation with constant forcings. It is clear for the greenhouse gas forcings. However, I was wondering whether any additional input files are needed to be specified. Also, which echam.scenario would I have to choose?
2. Fixing the issue would probably only allow me to run the 2100 until the end, but not beyond that.
3. Yes, that would indeed be what I had in mind. I might use some inspiration from @PengyangSong s reply below to approach this.

[PengyangSong]

I changed jsbach.datasets.yaml as below:

hist_fold: hist_harvest

# CD: Only for LU_hist definition is for years larger than 2014 (needed only to load
# forcing data of the current year + 1 to do an interpolation). The historical
# simulation in CMIP6 ends in 2014, the only data that is needed by echam is data from
# 2015 and we agreed to choose the one from the ssp370 scenario (it does not really
# matter which is chosen)
hist_extend_scenario: ssp370

input_sources:
        lctlib: "${echam.model_dir}/lctlib_nlct21.def"
        "jsbach_1850": "${adj_input_dir}/jsbach_${echam.resolution}${echam.ocean_resolution}_11tiles_5layers_1850${jsbachnc_file_ending}.nc"
        "jsbach_1976": "${adj_input_dir}/jsbach_${echam.resolution}${echam.ocean_resolution}_11tiles_5layers_1976${jsbachnc_file_ending}.nc"

        # FOCI specific input sources
        "cpools_foci": "${adj_input_dir}/cpools_${echam.resolution}${echam.ocean_resolution}_pdv0002_51251231.nc"
        "npools_foci": "${adj_input_dir}/cpools_${echam.resolution}${echam.ocean_resolution}_pdv0002_51251231.nc"
        "fpc_foci": "${adj_input_dir}/cpools_${echam.resolution}${echam.ocean_resolution}_pdv0002_51251231.nc"
        "lctlib_foci": "${computer.pool_directories.focipool}/ECHAM6_GENERAL/JSBACH/input/lctlib_nlct21.def"

forcing_sources:
        # harvest
        "LU_1850": "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${hist_fold}/LUH_harvest_${echam.resolution}_1850.nc"
        "LU_1950": "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${hist_fold}/LUH_harvest_${echam.resolution}_1950.nc"
        "LU_HIST":
                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${hist_fold}/LUH_harvest_${echam.resolution}_1850.nc":
                        to: 1850
                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${hist_fold}/LUH_harvest_${echam.resolution}_@[email protected]":
                        from: 1851
                        to: 2014
                # HIST scenario will use this data beyond year 2014
                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${hist_extend_scenario}_harvest/LUH_harvest_${echam.resolution}_@[email protected]":
                        from: 2015

        # landuse transitions
        "LU_trans_none": "${input_dir}/${echam.resolution}/New_Hampshire_LCC/no_LUH_transitions_${echam.resolution}.nc"
        "LU_trans_1850": "${input_dir}/${echam.resolution}/New_Hampshire_LCC/hist${jsbachnc_file_ending}/LUH_scaled_transitions_${echam.resolution}_1850.nc"
        "LU_trans_hist":
            "${input_dir}/${echam.resolution}/New_Hampshire_LCC/hist${jsbachnc_file_ending}/LUH_scaled_transitions_${echam.resolution}_@[email protected]":
                to: 2014
            # For the transient land data use a similar fix like in HIST land use
            "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${hist_extend_scenario}${jsbachnc_file_ending}/LUH_scaled_transitions_${echam.resolution}_@[email protected]":
                from: 2015

        # NOTE: these files are prepared by Christopher Danek (AWI). These are
        # yearly files where each file contain monthly data
        # Used by AWI-CM1 and AWI-CM1-REcoM in Levante for ssp scenarios
        Ndepo_ssp:
                # These files are 12-monthly; see README.md in Ndepo/ directory
                # NOTE: ssp370 might be missing in this directory
                "${input_dir_couple}/${Ndepo_path}@[email protected]":
                    from: 1850
                    to: 2099
                "${input_dir_couple}/${Ndepo_path}2099.nc":
                    from: 2100
                    to: 2201

choose_dataset:
        r0007:
                hist_fold: "hist"
                add_forcing_sources:
                        "cover_fract_HIST":
                                "${cover_fract_dir}/cover_fract_${echam.resolution}${echam.ocean_resolution}_11tiles_1850.nc":
                                        to: 1850
                                "${cover_fract_dir}/cover_fract_${echam.resolution}${echam.ocean_resolution}_11tiles_@[email protected]":
                                        from: 1851
        r0009:
                add_forcing_sources:
                        "Ndepo_1850": "${input_dir}/${echam.resolution}/${echam.resolution}_ndepo_CMIP_NCAR_CCMI-1-0_gr_185001-185012-clim.nc"
                        "a_nd_file": "${input_dir}/${echam.resolution}/spitfire/a_nd_${echam.resolution}.nc"
                        "lightning": "${input_dir}/${echam.resolution}/spitfire/LISOTD_HRMC_V2.2_${echam.resolution}.nc"
                        "popdens":   "${input_dir}/${echam.resolution}/spitfire/population_density_HYDE_${echam.resolution}.nc"
                        "cover_fract_HIST":
                                "${cover_fract_dir}/cover_fract_${echam.resolution}${echam.ocean_resolution}_11tiles_1850.nc":
                                        to: 1850
                                "${cover_fract_dir}/cover_fract_${echam.resolution}${echam.ocean_resolution}_11tiles_@[email protected]":
                                        from: 1851
        r0010:
                add_forcing_sources:
                        "Ndepo_1850": "${input_dir}/${echam.resolution}/${echam.resolution}_ndepo_CMIP_NCAR_CCMI-1-0_gr_185001-185012-clim.nc" # 12 months
                        "Ndepo_1950": "${input_dir}/${echam.resolution}/ndepo/Ndepo.1950.nc"
                        "a_nd_file": "${input_dir}/${echam.resolution}/spitfire/a_nd_${echam.resolution}.nc"
                        "lightning": "${input_dir}/${echam.resolution}/spitfire/LISOTD_HRMC_V2.2_${echam.resolution}.nc"
                        "popdens": "${input_dir}/${echam.resolution}/spitfire/population_density_HYDE_${echam.resolution}.nc" # ndecades
                        # Files used by AWI-CM1 and AWI-CM1-REcoM in Levante for ssp scenarios
                        "popdens_ssp":
                                "${input_dir_couple}/${echam.resolution}/spitfire/population_density_${ssp_num}_${echam.resolution}.nc": # see README.md in this dir
                                        from: 1850
                                        to: 2200
                        "cover_fract_HIST":
                                "${cover_fract_dir}/cover_fract_${echam.resolution}_11tiles_1850.nc":
                                        to: 1850
                                "${cover_fract_dir}/cover_fract_${echam.resolution}_11tiles_@[email protected]":
                                        from: 1851
                        "cover_fract_hist_scenario":
                                "${cover_fract_dir}/cover_fract_11tiles_1850.nc":
                                        to: 1850
                                "${cover_fract_dir}/cover_fract_11tiles_@[email protected]":
                                        from: 1851
                                        to: 2014
                                "${cover_fract_dir}/${echam.scenario}/cover_fract_11tiles_@[email protected]":
                                        from: 2015
                        "LU_trans_hist_scenario":
                                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/hist${dynveg}/LUH_scaled_transitions_${echam.resolution}_@[email protected]":
                                        to: 2014
                                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${echam.scenario}${dynveg}/LUH_scaled_transitions_${echam.resolution}_@[email protected]":
                                        from: 2015
                                        to: 2099
                                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${echam.scenario}${dynveg}/LUH_scaled_transitions_${echam.resolution}_2099.nc":
                                        from: 2100
                        "cover_fract_ssp":
                                "${cover_fract_dir}/cover_fract_${echam.resolution}_11tiles_1850.nc":
                                        to: 2014
                                "${input_dir}/${echam.resolution}/land_use_maps/${echam.scenario}/cover_fract_${echam.resolution}_11tiles_@[email protected]":
                                        from: 2015
                                        to: 2300

                        "LU_hist_scenario":
                                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/hist_harvest/LUH_harvest_${echam.resolution}_1850.nc":
                                        to: 1850
                                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/hist_harvest/LUH_harvest_${echam.resolution}_@[email protected]":
                                        from: 1851
                                        to: 2014
                                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${echam.scenario}_harvest/LUH_harvest_${echam.resolution}_@[email protected]":
                                        from: 2015
                                        to: 2099
                                "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${echam.scenario}_harvest/LUH_harvest_${echam.resolution}_2099.nc":
                                        from: 2100

                        "Ndepo_hist_scenario":
                        # TODO: pool on Levante seems to be broken currently.
                        # There are no yearly Ndepo files. ${input_dir_couple}
                        # contains the yearly files
                                "${input_dir}/${echam.resolution}/ndepo/Ndepo.1850.nc":
                                        to: 1850
                                "${input_dir}/${echam.resolution}/ndepo/Ndepo.@[email protected]":
                                        from: 1851
                                        to: 2014
                                # TODO: these files don't exist on Levante
                                "${input_dir}/${echam.resolution}/ndepo/Ndepo_${echam.scenario}.@[email protected]":
                                        from: 2015

I cannot attach the yaml file here. Try to compare with the one in your esm_tools to see the differences.

[jannitzbon]

Thanks @PengyangSong for this inspiration. I could use a similar approach for extending an SSP scenario beyond 2100. However, I am not sure for how many and for which input datasets (JSBACH, EHCAM, ...), I would need to implement such a case handling for after 2100.

[PengyangSong]

If I remember correct,
ECHAM is not the problem, greenhouse gas forcing is until 2500.
JSBACH datasets such as land use (LU), Ndepo, population density, cover frac ... needs extrapolation.

[crodehacke]

The suggested approach works nicely for the idealized scenarios (1pct4CO2 and 4CO2), where we performed simulations past 2775. If implemented, the missing files will be created in the directory $EXPID/forcing/jsbach, for instance, and fetch from there for the continuous simulation.

Good luck!

[chrisdane]

Hi Jan

Sorry for the late reply. To me thats two problems.

1. If you run into Time out of bounds in set_time_weight then the same echam src code must be fixed as described in ssp awicm recom #567, but for years > 2100. But if crodehacke utilized echam past 2775 already then their code version probably already covers this case?

2. Forcing files: you could simply make a check run with initial year 2101 restarted from some 2100 result and see which (forcing) files are missing. Just make sure that years > 2100 are covered by the

from: yyyy
to: yyyy

feature of the esm_tools as in Pengyangs example above. For testing you can use your own e.g. input_dir or cover_fract_dir by soft-linking to the default one and adding files you need. You can do these tests without defining a new echam.scenario: sspXXX (if everything works it would probably make sense to do so though).

Cheers,
Chris

[jannitzbon]

Hi again (@chrisdane @mandresm). I am still fighting with that Time out of bounds in set_time_weight error at the end of the year 2100. I tried to fix it in mo_simple-plumes.f90 as suggested above.
However, I am now facing the problem, that despite recompiling the code, the ECHAM is apparently not using the modified version of the source code for compilation. I checked that by changing error message, but it is still the old one. (I did create a fresh new run to make sure all binaries are updated.)
At the same time, I also did some modifications to JSBACH, and these are working perfectly fine after recompilation. So, I am wondering whether there is anything special about recompiling ECHAM or certain modules which has to be considered here?

[chrisdane]

Echam changes are ignored but Jsbach changes work? That is weird since usually both are compiled together. What compile command did you use and where?

[jannitzbon]

The modified code is here: /home/jnitzbon/remote/albedo/albedo/home/jnitzbon/git/esm_tools/awiesm-2.6
I once used the script comp-echam-6.3.05p2-awiesm-2.1_script.sh for compilation, and later also esm_master clean-awiesm-2.6 and then esm_master comp-awiesm-2.6 to recompile the whole model. All binaries in awiesm-2.6/bin were updated and also the *.o files recompiled.

Could there be any cached files/binaries which are used instead?

The experiment which fails to finish year 2100 is located here: /albedo/work/projects/p_paleo_ollie/jnitzbon/PermafrostCloudMIP/experiments/awiesm-2.6/PCMIP_SSP585_continued/

[jannitzbon]

Okay, I thin I found the issue with the compilation. I was modifying the file .../src/echam/mo_simple_plumes.f90, while the routine actually called is in the (duplicated?) source file .../src/rad_src/mo_simple_plumes.f90.

[chrisdane]

Nice that you figured that out. That is different from my version, where I had to modify .../src/mo_simple_plumes.f90.

Regarding

Would you suggest to adapt it such that the values for 2100 are taken for all years >2100? For example, by changing line 246 to:
iyear = MIN( MERGE(FLOOR(year_fr), fix_yr, fix_yr == -9999) - 1849, nyears)

Yes, I think thats the way I would go without checking the details now.

[mandresm]

Hi! Thank you all for your contributions. This starts to feel like a nicely connected community :)

Ineed, as @PengyangSong mentioned, the from to logic is the way to go for the forcings issue.

I would add to this, that it is best if you add this syntax directly to your runscript as:

jsbach:
    add_forcing_sources:
        "${input_dir}/${echam.resolution}/New_Hampshire_LCC/${echam.scenario}_harvest/LUH_harvest_${echam.resolution}_2099.nc":
            from: 2100

meaning, any change you would have to do in the jsbach.dataset.yaml put it on the runscript. I am recommending this because then all the changes that you did to the main repo to run your experiment are in your runscript, which basically ensures that if you run it again in the future with a new version of ESM-Tools that does not have these changes, it can still be reproducible, or you can just share it with a colleague without them needing to change branch or copy your other yamls.