E3SM-Project · chengzhuzhang · Oct 7, 2025 · Oct 6, 2025 · Oct 6, 2025 · Oct 7, 2025
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -18,6 +18,7 @@ simulations.
    v1/index
    v2/index
    v2.1/index
+   v3/index
    SCREAMv0/index
    SCREAMv1/index
 

diff --git a/docs/source/v2/WaterCycle/simulation_data/index.rst b/docs/source/v2/WaterCycle/simulation_data/index.rst
@@ -42,6 +42,12 @@ For more information, refer to `zstash usage <https://e3sm-project.github.io/zst
 `web interface <https://portal.nersc.gov/archive/home/projects/e3sm/www/WaterCycle/E3SMv2>`_.
 Note that this will be slow and inefficient since you'll have to download the tar files.
 
+**Tip for users without NERSC access**: Before downloading large tar files, you can first download the ``index.db`` file and use sqlite to check the archive contents: ::
+
+   sqlite3 index.db "SELECT tar,name,size from files;" > archive_contents.txt
+
+This will help you identify which specific tar files contain the data you need before downloading.
+
 **v2.LR** simulations data has been archived on NERSC HPSS under: ::
 
   /home/projects/e3sm/www/WaterCycle/E3SMv2/LR

diff --git a/docs/source/v3/CoupledSystem/index.rst b/docs/source/v3/CoupledSystem/index.rst
@@ -0,0 +1,21 @@
+E3SMv3 
+================
+
+The simulation campaign for E3SMv3  was performed initially with the
+**v3.LR** (lower resolution) model configuration.
+
+If you use data from this simulation campaign, please cite the relevant overview
+manuscripts.
+
+* For a general E3SMv3 overview and v3.LR simulations:
+
+  * Xie et al. (2025). The Energy Exascale Earth System Model Version 3. Part I: Overview of the Atmospheric Component. Accepted by JAMES. DOI to follow.
+  * Golaz et al. (2025). The Energy Exascale Earth System Model Version 3. Part II: Overview of the Coupled system. Submitted to JAMES.
+
+For information on how to access and use the simulation data, see :doc:`here <../../v2/WaterCycle/simulation_data/index>`.
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   simulation_data/index
diff --git a/docs/source/v3/CoupledSystem/simulation_data/index.rst b/docs/source/v3/CoupledSystem/simulation_data/index.rst
@@ -0,0 +1,7 @@
+Simulation Data
+===============
+
+.. toctree::
+   :maxdepth: 2
+
+   simulation_table
diff --git a/docs/source/v3/CoupledSystem/simulation_data/simulation_table.rst b/docs/source/v3/CoupledSystem/simulation_data/simulation_table.rst
diff --git a/docs/source/v3/index.rst b/docs/source/v3/index.rst
@@ -0,0 +1,8 @@
+v3.LR
+==========================================
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents:
+
+   CoupledSystem/index
diff --git a/machine_readable_data/v3_CoupledSystem_simulations.csv b/machine_readable_data/v3_CoupledSystem_simulations.csv
@@ -0,0 +1,31 @@
+Simulation,Data Size (TB),ESGF Links,HPSS Path,HPSS URL
+v3.LR.piControl,69,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piControl,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piControl
+v3.LR.abrupt-4xCO2_0101_bcdt15m,15,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.abrupt-4xCO2_0101_bcdt15m,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.abrupt-4xCO2_0101_bcdt15m
+v3.LR.1pctCO2_0101_bcdt15m,16,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.1pctCO2_0101_bcdt15m,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.1pctCO2_0101_bcdt15m
+v3.LR.historical_0051,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0051,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0051
+v3.LR.historical_0101,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0101,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0101
+v3.LR.historical_0151,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0151,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0151
+v3.LR.historical_0201,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0201,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0201
+v3.LR.historical_0251,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0251,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.historical_0251
+v3.LR.hist-GHG_0101,20,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-GHG_0101,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-GHG_0101
+v3.LR.hist-GHG_0151,20,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-GHG_0151,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-GHG_0151
+v3.LR.hist-GHG_0201,20,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-GHG_0201,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-GHG_0201
+v3.LR.hist-aer_0101,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-aer_0101,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-aer_0101
+v3.LR.hist-aer_0151,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-aer_0151,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-aer_0151
+v3.LR.hist-aer_0201,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-aer_0201,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-aer_0201
+v3.LR.hist-xGHG-xaer_0101,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-xGHG-xaer_0101,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-xGHG-xaer_0101
+v3.LR.hist-xGHG-xaer_0151,20,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-xGHG-xaer_0151,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-xGHG-xaer_0151
+v3.LR.hist-xGHG-xaer_0201,19,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-xGHG-xaer_0201,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.hist-xGHG-xaer_0201
+v3.LR.amip_0101,8,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.amip_0101,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.amip_0101
+v3.LR.amip_0151,6,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.amip_0151,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.amip_0151
+v3.LR.amip_0201,6,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.amip_0201,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.amip_0201
+v3.LR.piClim-control-iceini,3,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-control-iceini,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-control-iceini
+v3.LR.piClim-histall/v3.LR.piClim-histall_0101,9,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histall/v3.LR.piClim-histall_0101,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histall/v3.LR.piClim-histall_0101
+v3.LR.piClim-histall/v3.LR.piClim-histall_0151,9,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histall/v3.LR.piClim-histall_0151,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histall/v3.LR.piClim-histall_0151
+v3.LR.piClim-histall/v3.LR.piClim-histall_0201,9,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histall/v3.LR.piClim-histall_0201,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histall/v3.LR.piClim-histall_0201
+v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0101,9,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0101,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0101
+v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0151,9,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0151,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0151
+v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0201,9,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0201,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0201
+v3.LR.piClim-histaer/v3.LR.piClim-histaer_0101,8,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histaer/v3.LR.piClim-histaer_0101,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histaer/v3.LR.piClim-histaer_0101
+v3.LR.piClim-histaer/v3.LR.piClim-histaer_0151,8,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histaer/v3.LR.piClim-histaer_0151,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histaer/v3.LR.piClim-histaer_0151
+v3.LR.piClim-histaer/v3.LR.piClim-histaer_0201,9,,/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histaer/v3.LR.piClim-histaer_0201,https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piClim-histaer/v3.LR.piClim-histaer_0201
diff --git a/utils/generate_tables.py b/utils/generate_tables.py
@@ -173,6 +173,9 @@ def __init__(self, simulation_dict):
 
         self.esgf = get_esgf(self.model_version, self.resolution, self.simulation_name, self.experiment, self.ensemble_num, self.link_type, self.node)
 
+        # Generate web interface URL from HPSS path
+        self.web_interface = self.get_web_interface_url()
+
         self.run_script_original = get_run_script_original(self.model_version, self.simulation_name)
         self.run_script_reproduction = get_run_script_reproduction(self.model_version, self.simulation_name)
 
@@ -183,9 +186,20 @@ def __init__(self, simulation_dict):
         if not self.run_script_original:
             self.run_script_original = "N/A"
 
+    def get_web_interface_url(self) -> str:
+        """Generate web interface URL from HPSS path"""
+        if self.hpss and self.data_size:
+            # Convert HPSS path to web interface URL
+            # /home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piControl -> https://portal.nersc.gov/archive/home/projects/e3sm/www/CoupledSystem/E3SMv3/LR/v3.LR.piControl
+            hpss_clean = self.hpss.replace("(symlink) ", "")  # Remove symlink prefix if present
+            # Use the full path - each simulation gets its own distinct URL
+            web_url = f"https://portal.nersc.gov/archive{hpss_clean}"
+            return f"`HPSS URL <{web_url}>`_"
+        return ""
+
     def get_row(self, output_file, minimal_content: bool = False) -> List[str]:
         if "simulation" in output_file:
-            row = [self.simulation_name, self.data_size, self.esgf, self.hpss]
+            row = [self.simulation_name, self.data_size, self.esgf, self.hpss, self.web_interface]
             if minimal_content:
                 match_object: re.Match = re.match("`.*<(.*)>`_", self.esgf)
                 if match_object:
@@ -195,6 +209,10 @@ def get_row(self, output_file, minimal_content: bool = False) -> List[str]:
                     # Since we don't want that in the csv output,
                     # which a computer reads.
                     row[3] = row[3].replace("(symlink) ", "")
+                # Extract web interface URL for CSV
+                web_match: re.Match = re.match("`.*<(.*)>`_", self.web_interface)
+                if web_match:
+                    row[4] = web_match.group(1)  # Extract URL from the web interface link
             return row
         elif "reproduction" in output_file:
             return [self.simulation_name, self.machine, self.checksum, self.run_script_reproduction, self.run_script_original]
@@ -261,6 +279,8 @@ def read_simulations(csv_file):
                         simulation_dict["link_type"] = "both"
                     elif simulation_dict["cmip_only"] == "cmip_only":
                         simulation_dict["link_type"] = "cmip"
+                    elif simulation_dict["cmip_only"] == "none":
+                        simulation_dict["link_type"] = "none"
                     else:
                         raise ValueError(f"Invalid cmip_only={simulation_dict['cmip_only']}")
                 simulation_dicts.append(simulation_dict)
@@ -324,6 +344,10 @@ def pad_cells_row_dividers(marker: str, cell_paddings: List[int]) -> str:
     return string
 
 def generate_table(page_type: str, resolutions: OrderedDict[str, Category], header_cells: List[str], output_file: str, cell_paddings: List[int]):
+
+    # Create output directory if it doesn't exist
+    os.makedirs(os.path.dirname(output_file), exist_ok=True)
+
     with open(output_file, "w") as file_write:
         # Page Title
         file_write.write("**********************************\n")
@@ -349,7 +373,7 @@ def generate_table(page_type: str, resolutions: OrderedDict[str, Category], head
 def construct_pages(csv_file: str, model_version: str, group_name: str, include_reproduction_scripts: bool = False):
     versions: OrderedDict[str, ModelVersion] = read_simulations(csv_file)
     resolutions: OrderedDict[str, Category] = versions[model_version].groups[group_name].resolutions
-    header_cells: List[str] = ["Simulation", "Data Size (TB)", "ESGF Links", "HPSS Path"]
+    header_cells: List[str] = ["Simulation", "Data Size (TB)", "ESGF Links", "HPSS Path", "HPSS URL"]
     construct_output_csv(resolutions, header_cells, f"../machine_readable_data/{model_version}_{group_name}_simulations.csv")
     print(f"csv of the simulations will be available at https://github.com/E3SM-Project/e3sm_data_docs/blob/main/machine_readable_data/{model_version}_{group_name}_simulations.csv")
     # TODO: add proper subdirs and index.rst files in docs/
@@ -358,7 +382,7 @@ def construct_pages(csv_file: str, model_version: str, group_name: str, include_
         resolutions,
         header_cells,
         f"../docs/source/{model_version}/{group_name}/simulation_data/simulation_table.rst",
-        [85, 15, 400, 140]
+        [65, 15, 220, 120, 140]
     )
     if include_reproduction_scripts:
         header_cells_reproduction: List[str] = ["Simulation", "Machine", "10 day checksum", "Reproduction Script", "Original Script (requires significant changes to run!!)",]
@@ -383,4 +407,5 @@ def construct_pages(csv_file: str, model_version: str, group_name: str, include_
     # Sources for v1 data
     # https://acme-climate.atlassian.net/wiki/spaces/ED/pages/4495441922/V1+Simulation+backfill+WIP
     # https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/1271169273/v1+High+Res+Coupled+Run+Output+HPSS+Archive 
-    construct_pages("input/simulations_v1_water_cycle.csv", "v1", "WaterCycle")
+    # construct_pages("input/simulations_v1_water_cycle.csv", "v1", "WaterCycle")
+    construct_pages("input/simulations_v3_LR_coupled.csv", "v3", "CoupledSystem")
diff --git a/utils/input/simulations_v3_LR_coupled.csv b/utils/input/simulations_v3_LR_coupled.csv
@@ -0,0 +1,31 @@
+model_version, group, resolution, category, simulation_name, machine, checksum, experiment, ensemble_num, cmip_only,
+v3, CoupledSystem, LR, DECK, v3.LR.piControl, chrysalis, , piControl, 1, none,
+v3, CoupledSystem, LR, DECK, v3.LR.abrupt-4xCO2_0101_bcdt15m, chrysalis, , abrupt-4xCO2, 1, none,
+v3, CoupledSystem, LR, DECK, v3.LR.1pctCO2_0101_bcdt15m, chrysalis, , 1pctCO2, 1, none,
+v3, CoupledSystem, LR, Historical, v3.LR.historical_0051, chrysalis, , historical, 1, none,
+v3, CoupledSystem, LR, Historical, v3.LR.historical_0101, chrysalis, , historical, 2, none,
+v3, CoupledSystem, LR, Historical, v3.LR.historical_0151, chrysalis, , historical, 3, none,
+v3, CoupledSystem, LR, Historical, v3.LR.historical_0201, chrysalis, , historical, 4, none,
+v3, CoupledSystem, LR, Historical, v3.LR.historical_0251, chrysalis, , historical, 5, none,
+v3, CoupledSystem, LR, Single-forcing (DAMIP-like), v3.LR.hist-GHG_0101, chrysalis, , hist-GHG, 1, none,
+v3, CoupledSystem, LR, Single-forcing (DAMIP-like), v3.LR.hist-GHG_0151, chrysalis, , hist-GHG, 2, none,
+v3, CoupledSystem, LR, Single-forcing (DAMIP-like), v3.LR.hist-GHG_0201, chrysalis, , hist-GHG, 3, none,
+v3, CoupledSystem, LR, Single-forcing (DAMIP-like), v3.LR.hist-aer_0101, chrysalis, , hist-aer, 1, none,
+v3, CoupledSystem, LR, Single-forcing (DAMIP-like), v3.LR.hist-aer_0151, chrysalis, , hist-aer, 2, none,
+v3, CoupledSystem, LR, Single-forcing (DAMIP-like), v3.LR.hist-aer_0201, chrysalis, , hist-aer, 3, none,
+v3, CoupledSystem, LR, Single-forcing (DAMIP-like), v3.LR.hist-xGHG-xaer_0101, chrysalis, , hist-nat, 1, none,
+v3, CoupledSystem, LR, Single-forcing (DAMIP-like), v3.LR.hist-xGHG-xaer_0151, chrysalis, , hist-nat, 2, none,
+v3, CoupledSystem, LR, Single-forcing (DAMIP-like), v3.LR.hist-xGHG-xaer_0201, chrysalis, , hist-nat, 3, none,
+v3, CoupledSystem, LR, AMIP, v3.LR.amip_0101, chrysalis, , amip, 1, none,
+v3, CoupledSystem, LR, AMIP, v3.LR.amip_0151, chrysalis, , amip, 2, none,
+v3, CoupledSystem, LR, AMIP, v3.LR.amip_0201, chrysalis, , amip, 3, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-control-iceini, chrysalis, , piClim-control, 1, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-histall/v3.LR.piClim-histall_0101, chrysalis, , piClim-histall, 1, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-histall/v3.LR.piClim-histall_0151, chrysalis, , piClim-histall, 2, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-histall/v3.LR.piClim-histall_0201, chrysalis, , piClim-histall, 3, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0101, chrysalis, , piClim-histGHG, 1, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0151, chrysalis, , piClim-histGHG, 2, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-histGHG/v3.LR.piClim-histGHG_0201, chrysalis, , piClim-histGHG, 3, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-histaer/v3.LR.piClim-histaer_0101, chrysalis, , piClim-histaer, 1, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-histaer/v3.LR.piClim-histaer_0151, chrysalis, , piClim-histaer, 2, none,
+v3, CoupledSystem, LR, RFMIP, v3.LR.piClim-histaer/v3.LR.piClim-histaer_0201, chrysalis, , piClim-histaer, 3, none,