Skip to content

Reduce requested time, memory, and CPU allocation in benchmark and integration test scripts #3275

@yantosca

Description

@yantosca

Your name

Bob Yantosca

Your affiliation

Harvard + GCST

Provide a clear and concise overview of the new feature requested.

Our benchmark run scripts and integration test scripts are currently over-requesting memory and CPUs. Here is the summary of the integration test compile process for GC-Classic:

                              Overall Utilization                               
================================================================================
  CPU utilization  [||||||||||||||                                 29%]
  CPU memory usage [||||||||||||||||||||||||||                     52%]

                              Detailed Utilization                              
================================================================================
  CPU utilization per node (CPU time used/run time)
      holy7c02507: 03:22:31/11:19:52 (efficiency=29.8%)

  CPU memory usage per node - used/allocated
      holy7c02507: 3.0GB/5.9GB (388.5MB/750.0MB per core of 8)

and for the integration test run script:

                              Overall Utilization                               
================================================================================
  CPU utilization  [||||||||||||||||||||||||||||||||||||||         77%]
  CPU memory usage [|||||||||||||||||||||||||                      50%]

                              Detailed Utilization                              
================================================================================
  CPU utilization per node (CPU time used/run time)
      holy7c24412: 1-00:05:58/1-07:10:48 (efficiency=77.3%)

  CPU memory usage per node - used/allocated
      holy7c24412: 72.6GB/146.5GB (3.0GB/6.1GB per core of 24)

We can certainly reduce the amount of requested time, CPUs, and memory in order to make the jobs more efficient. This might involve a bit of testing but would be worth it.

Reference(s)

No response

Will you be implementing this feature yourself?

Yes

Additional information

No response

Metadata

Metadata

Assignees

Labels

category: Feature RequestNew feature or requestno-diff-to-benchmarkThis update will not change the results of fullchem benchmark simulationstopic: Benchmarking and TestingRelated to CI, integration tests, or scientific benchmarkingtopic: Structural ModificationsRelated to GEOS-Chem structural modifications (as opposed to scientific updates)

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions