Skip to content

Commit bc2b2d0

Browse files
baagaard-usgsbaagaard-usgs
authored
Merge pull request #859 from baagaard-usgs/feature-adaptive-ts-main
Add adaptive time stepping parameters to PETSc options. Illustrate adaptive time stepping in examples.
2 parents 28bbf74 + 8f642e6 commit bc2b2d0

File tree

65 files changed

+4570
-438
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

65 files changed

+4570
-438
lines changed

‎CHANGES.md‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,14 @@ The version numbers are in the form `MAJOR.MINOR.PATCH`, where major releases in
88
## Version 5.0.0
99

1010
* **Changed**
11+
* Update the nondimensionalization code, moving it from SpatialData to PyLith.
12+
* Rename Nondimensional to Scales.
13+
* Add displacement scale and improve the names of the other scales.
14+
* The time scale for poroelasticity is derived from the other scales and nominal material properties.
15+
* Add discussion of nondimensionalization for each of the governing equations.
16+
* Update default solver tolerances consistent with new nondimensionalization scales.
17+
* Add option to use adaptive time stepping and update examples `reverse-2d` Steps 7 and 8 and `magma-2d` Step 1 to demonstrate its use.
18+
* Update solver settings for elasticity for better scalability with problem size.
1119
* ASCII mesh format
1220
* Changed `group` to `vertex-group`; remove `group` `type`.
1321
* Renamed `VertexGroup` to `BoundaryGroup` in `meshio.gmsh_utils` and changed default behavior to not be recursive (generate "face" groups, not "vertex" groups).

‎docs/Makefile.am‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -333,6 +333,8 @@ dist_noinst_DATA = \
333333
user/examples/magma-2d/figs/step01-diagram.svg \
334334
user/examples/magma-2d/figs/step01-solution.jpg \
335335
user/examples/magma-2d/figs/step02-solution.jpg \
336+
user/examples/magma-2d/figs/step01-compare.pdf \
337+
user/examples/magma-2d/figs/step01-compare.svg \
336338
user/examples/poroelastic-outerrise-2d/index.md \
337339
user/examples/poroelastic-outerrise-2d/meshing-gmsh.md \
338340
user/examples/poroelastic-outerrise-2d/common-information.md \
@@ -422,7 +424,11 @@ dist_noinst_DATA = \
422424
user/examples/reverse-2d/figs/step06-diagram.svg \
423425
user/examples/reverse-2d/figs/step06-solution.jpg \
424426
user/examples/reverse-2d/figs/step07-solution.jpg \
427+
user/examples/reverse-2d/figs/step07-compare.pdf \
428+
user/examples/reverse-2d/figs/step07-compare.svg \
425429
user/examples/reverse-2d/figs/step08-solution.jpg \
430+
user/examples/reverse-2d/figs/step08-compare.pdf \
431+
user/examples/reverse-2d/figs/step08-compare.svg \
426432
user/examples/strikeslip-2d/index.md \
427433
user/examples/strikeslip-2d/common-information.md \
428434
user/examples/strikeslip-2d/exercises.md \

‎docs/user/examples/box-2d/step01-axialdisp.md‎

Lines changed: 25 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -75,54 +75,64 @@ $ pylith step01_axialdisp.cfg
7575
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:221:_printInfo
7676
-- timedependent(info)
7777
-- Scales for nondimensionalization:
78-
Length scale: 1000*m
78+
Length scale: 100000*m
79+
Displacement scale: 1*m
7980
Time scale: 3.15576e+09*s
80-
Pressure scale: 3e+10*m**-1*kg*s**-2
81-
Density scale: 2.98765e+23*m**-3*kg
81+
Rigidity scale: 1e+10*m**-1*kg*s**-2
8282
Temperature scale: 1*K
8383
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:186:initialize
8484
-- timedependent(info)
8585
-- Initializing timedependent problem with quasistatic formulation.
8686
>> /src/cig/pylith/libsrc/pylith/utils/PetscOptions.cc:235:static void pylith::utils::_PetscOptions::write(pythia::journal::info_t &, const char *, const pylith::utils::PetscOptions &)
8787
-- petscoptions(info)
8888
-- Setting PETSc options:
89-
ksp_atol = 1.0e-12
89+
ksp_atol = 1.0e-7
9090
ksp_converged_reason = true
9191
ksp_error_if_not_converged = true
92+
ksp_guess_pod_size = 8
93+
ksp_guess_type = pod
9294
ksp_rtol = 1.0e-12
93-
pc_type = lu
94-
snes_atol = 1.0e-9
95+
mg_fine_ksp_max_it = 5
96+
mg_levels_pc_type = pbjacobi
97+
pc_gamg_coarse_eq_limit = 200
98+
pc_type = gamg
99+
snes_atol = 4.0e-7
95100
snes_converged_reason = true
96101
snes_error_if_not_converged = true
97102
snes_monitor = true
98103
snes_rtol = 1.0e-12
99104
ts_error_if_step_fails = true
105+
ts_exact_final_time = matchstep
100106
ts_monitor = true
101107
ts_type = beuler
108+
viewer_hdf5_collective = true
102109
103110
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/TimeDependent.py:139:run
104111
-- timedependent(info)
105112
-- Solving problem.
106-
0 TS dt 0.01 time 0.
107-
0 SNES Function norm 1.245882095312e-02
108-
Linear solve converged due to CONVERGED_ATOL iterations 1
109-
1 SNES Function norm 6.738354969624e-18
110-
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
111-
1 TS dt 0.01 time 0.01
113+
0 TS dt 0.001 time 0.
114+
0 SNES Function norm 3.737646285937e+01
115+
Linear solve converged due to CONVERGED_ATOL iterations 4
116+
1 SNES Function norm 2.117345506899e-09
117+
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
118+
1 TS dt 0.001 time 0.001
112119
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:201:finalize
113120
-- timedependent(info)
114121
-- Finalizing problem.
115-
```
122+
WARNING! There are options you set that were not used!
123+
WARNING! could be spelling mistake, etc!
124+
There is one unused database option. It is:
125+
Option left: name:-mg_levels_pc_type value: pbjacobi source: code```
116126
117127
At the beginning of the output written to the terminal, we see that PyLith is reading the mesh using the `MeshIOAscii` reader and that it found the domain to extend from -6000 m to +6000 m in the x direction and from -16000 m to 0 m in the y direction.
118128
We also see the scales used to nondimensionalize the problem.
119-
The density scale is very large for quasistatic problems.
120129
121130
Near the end of the output written to the terminal, we see the PETSc options PyLith selected based on the governing equations and formulation as discussed in {ref}`sec-user-run-pylith-petsc-options`.
122131
The solver advanced the solution one time step (static simulation).
123-
The linear solve converged in 1 iteration, consistent with the LU preconditioner.
132+
The linear solve converged in 4 iterations.
124133
The norm of the residual met the absolute tolerance convergence criterion (`ksp_atol`).
125134
The nonlinear solve converged in 1 iteration, which we expect because this is a linear problem, and the residual met the absolute convergence tolerance (`snes_atol`).
135+
For this small problem, the multi-grid preconditioner has fewer levels, so we get a warning about an unused PETSc option.
126136
127137
## Visualizing the results
128138

‎docs/user/examples/box-2d/step02-sheardisp.md‎

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -65,16 +65,19 @@ $ pylith step02_sheardisp.cfg
6565
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/TimeDependent.py:139:run
6666
-- timedependent(info)
6767
-- Solving problem.
68-
0 TS dt 0.01 time 0.
69-
0 SNES Function norm 2.239977678460e-03
70-
Linear solve converged due to CONVERGED_ATOL iterations 1
71-
1 SNES Function norm 1.964321818484e-18
72-
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
73-
1 TS dt 0.01 time 0.01
68+
0 TS dt 0.001 time 0.
69+
0 SNES Function norm 6.719933035381e+00
70+
Linear solve converged due to CONVERGED_ATOL iterations 3
71+
1 SNES Function norm 1.640603910123e-07
72+
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
73+
1 TS dt 0.001 time 0.001
7474
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:201:finalize
7575
-- timedependent(info)
7676
-- Finalizing problem.
77-
```
77+
WARNING! There are options you set that were not used!
78+
WARNING! could be spelling mistake, etc!
79+
There is one unused database option. It is:
80+
Option left: name:-mg_levels_pc_type value: pbjacobi source: code```
7881
7982
The output written to the terminal is nearly identical to what we saw for Step 1.
8083
We omit the middle portion of the output which shows that the domain, the scales for nondimensionalization, and PETSc options all remain the same.

‎docs/user/examples/box-2d/step03-sheardisptract.md‎

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -76,19 +76,23 @@ $ pylith step03_sheardisptract.cfg
7676
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/TimeDependent.py:139:run
7777
-- timedependent(info)
7878
-- Solving problem.
79-
0 TS dt 0.01 time 0.
80-
0 SNES Function norm 6.059797141590e-03
81-
Linear solve converged due to CONVERGED_ATOL iterations 1
82-
1 SNES Function norm 2.140441363908e-18
83-
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
84-
1 TS dt 0.01 time 0.01
79+
0 TS dt 0.001 time 0.
80+
0 SNES Function norm 1.817939142477e+01
81+
Linear solve converged due to CONVERGED_ATOL iterations 3
82+
1 SNES Function norm 8.280473801872e-08
83+
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
84+
1 TS dt 0.001 time 0.001
8585
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:201:finalize
8686
-- timedependent(info)
8787
-- Finalizing problem.
88-
```
88+
WARNING! There are options you set that were not used!
89+
WARNING! could be spelling mistake, etc!
90+
There is one unused database option. It is:
91+
Option left: name:-mg_levels_pc_type value: pbjacobi source: code```
8992
9093
As expected, the output written to the terminal is nearly identical to what we saw for Steps 1 and 2.
9194
95+
9296
## Visualizing the results
9397
9498
In {numref}`fig:example:box:2d:step03:solution` we use the `pylith_viz` utility to visualize the x displacement field.

‎docs/user/examples/box-2d/step04-sheardispic.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,8 +61,9 @@ $ pylith step04_sheardispic.cfg
6161
-- Finalizing problem.
6262
WARNING! There are options you set that were not used!
6363
WARNING! could be spelling mistake, etc!
64-
There is one unused database option. It is:
65-
Option left: name:-ksp_converged_reason (no value)
64+
There are 2 unused database options. They are:
65+
Option left: name:-mg_fine_ksp_max_it value: 5 source: code
66+
Option left: name:-mg_levels_pc_type value: pbjacobi source: code
6667
```
6768

6869
By design we set the initial condition so that it satisfies the elasticity equation.

‎docs/user/examples/box-2d/step05-sheardisptractrate.md‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -86,10 +86,10 @@ $ pylith step05_sheardisptractrate.cfg
8686
# -- many lines omitted --
8787
8888
5 TS dt 0.1 time 0.4
89-
0 SNES Function norm 1.467261021331e-03
90-
Linear solve converged due to CONVERGED_ATOL iterations 1
91-
1 SNES Function norm 7.893110957891e-19
92-
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
89+
0 SNES Function norm 4.544847874296e+00
90+
Linear solve converged due to CONVERGED_ATOL iterations 0
91+
1 SNES Function norm 1.606117666093e-07
92+
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
9393
6 TS dt 0.1 time 0.5
9494
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:201:finalize
9595
-- timedependent(info)

‎docs/user/examples/box-3d/step01-axialdisp.md‎

Lines changed: 25 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -75,54 +75,65 @@ $ pylith step01_axialdisp.cfg
7575
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:221:_printInfo
7676
-- timedependent(info)
7777
-- Scales for nondimensionalization:
78-
Length scale: 1000*m
78+
Length scale: 100000*m
79+
Displacement scale: 1*m
7980
Time scale: 3.15576e+09*s
80-
Pressure scale: 3e+10*m**-1*kg*s**-2
81-
Density scale: 2.98765e+23*m**-3*kg
81+
Rigidity scale: 1e+10*m**-1*kg*s**-2
8282
Temperature scale: 1*K
8383
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:186:initialize
8484
-- timedependent(info)
8585
-- Initializing timedependent problem with quasistatic formulation.
8686
>> /src/cig/pylith/libsrc/pylith/utils/PetscOptions.cc:235:static void pylith::utils::_PetscOptions::write(pythia::journal::info_t &, const char *, const pylith::utils::PetscOptions &)
8787
-- petscoptions(info)
8888
-- Setting PETSc options:
89-
ksp_atol = 1.0e-12
89+
ksp_atol = 1.0e-7
9090
ksp_converged_reason = true
9191
ksp_error_if_not_converged = true
92+
ksp_guess_pod_size = 8
93+
ksp_guess_type = pod
9294
ksp_rtol = 1.0e-12
93-
pc_type = lu
94-
snes_atol = 1.0e-9
95+
mg_fine_ksp_max_it = 5
96+
mg_levels_pc_type = pbjacobi
97+
pc_gamg_coarse_eq_limit = 200
98+
pc_type = gamg
99+
snes_atol = 4.0e-7
95100
snes_converged_reason = true
96101
snes_error_if_not_converged = true
97102
snes_monitor = true
98103
snes_rtol = 1.0e-12
99104
ts_error_if_step_fails = true
105+
ts_exact_final_time = matchstep
100106
ts_monitor = true
101107
ts_type = beuler
108+
viewer_hdf5_collective = true
102109
103110
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/TimeDependent.py:139:run
104111
-- timedependent(info)
105112
-- Solving problem.
106-
0 TS dt 0.01 time 0.
107-
0 SNES Function norm 6.816802711276e-02
108-
Linear solve converged due to CONVERGED_ATOL iterations 1
109-
1 SNES Function norm 3.270841134292e-17
110-
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
111-
1 TS dt 0.01 time 0.01
113+
0 TS dt 0.001 time 0.
114+
0 SNES Function norm 2.045040813383e+00
115+
Linear solve converged due to CONVERGED_ATOL iterations 4
116+
1 SNES Function norm 3.894554358059e-09
117+
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
118+
1 TS dt 0.001 time 0.001
112119
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:201:finalize
113120
-- timedependent(info)
114121
-- Finalizing problem.
122+
WARNING! There are options you set that were not used!
123+
WARNING! could be spelling mistake, etc!
124+
There is one unused database option. It is:
125+
Option left: name:-mg_levels_pc_type value: pbjacobi source: code
115126
```
116127

117128
At the beginning of the output written to the terminal, we see that PyLith is reading the mesh using the `MeshIOAscii` reader and that it found the domain to extend from -6000 m to +6000 m in the x and y directions and from -9000 m to 0 m in the z direction.
118129
We also see the scales used to nondimensionalize the problem.
119-
The density scale is very large for quasistatic problems.
120130

121131
Near the end of the output written to the terminal, we see the PETSc options PyLith selected based on the governing equations and formulation as discussed in {ref}`sec-user-run-pylith-petsc-options`.
122132
The solver advanced the solution one time step (static simulation).
123-
The linear solve converged in 1 iteration, consistent with the LU preconditioner.
133+
The linear solve converged in 4 iterations.
124134
The norm of the residual met the absolute tolerance convergence criterion (`ksp_atol`).
125135
The nonlinear solve converged in 1 iteration, which we expect because this is a linear problem, and the residual met the absolute convergence tolerance (`snes_atol`).
136+
For this small problem, the multi-grid preconditioner has fewer levels, so we get a warning about an unused PETSc option.
126137

127138
## Visualizing the results
128139

‎docs/user/examples/box-3d/step02-sheardisp.md‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -67,15 +67,19 @@ $ pylith step02_sheardisp.cfg
6767
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/TimeDependent.py:139:run
6868
-- timedependent(info)
6969
-- Solving problem.
70-
0 TS dt 0.01 time 0.
71-
0 SNES Function norm 1.811215061775e-02
72-
Linear solve converged due to CONVERGED_ATOL iterations 1
73-
1 SNES Function norm 2.330640615892e-17
74-
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
75-
1 TS dt 0.01 time 0.01
70+
0 TS dt 0.001 time 0.
71+
0 SNES Function norm 5.433645185324e-01
72+
Linear solve converged due to CONVERGED_ATOL iterations 4
73+
1 SNES Function norm 4.280274905934e-09
74+
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
75+
1 TS dt 0.001 time 0.001
7676
>> /software/unix/py39-venv/pylith-debug/lib/python3.9/site-packages/pylith/problems/Problem.py:201:finalize
7777
-- timedependent(info)
7878
-- Finalizing problem.
79+
WARNING! There are options you set that were not used!
80+
WARNING! could be spelling mistake, etc!
81+
There is one unused database option. It is:
82+
Option left: name:-mg_levels_pc_type value: pbjacobi source: code
7983
```
8084

8185
The output written to the terminal is nearly identical to what we saw for Step 1.

‎docs/user/examples/box-3d/step03-sheardisptract.md‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -82,15 +82,19 @@ $ pylith step03_sheardisptract.cfg
8282
>> /software/unix/py38-venv/pylith-debug/lib/python3.8/site-packages/pylith/problems/TimeDependent.py:139:run
8383
-- timedependent(info)
8484
-- Solving problem.
85-
0 TS dt 0.01 time 0.
86-
0 SNES Function norm 2.854246293576e-02
87-
Linear solve converged due to CONVERGED_ATOL iterations 1
88-
1 SNES Function norm 2.511862662012e-17
89-
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
90-
1 TS dt 0.01 time 0.01
85+
0 TS dt 0.001 time 0.
86+
0 SNES Function norm 8.562738880728e-01
87+
Linear solve converged due to CONVERGED_ATOL iterations 4
88+
1 SNES Function norm 3.584100837616e-09
89+
Nonlinear solve converged due to CONVERGED_FNORM_ABS iterations 1
90+
1 TS dt 0.001 time 0.001
9191
>> /software/unix/py38-venv/pylith-debug/lib/python3.8/site-packages/pylith/problems/Problem.py:201:finalize
9292
-- timedependent(info)
9393
-- Finalizing problem.
94+
WARNING! There are options you set that were not used!
95+
WARNING! could be spelling mistake, etc!
96+
There is one unused database option. It is:
97+
Option left: name:-mg_levels_pc_type value: pbjacobi source: code
9498
```
9599

96100
The output written to the terminal is nearly identical to what we saw for Step 2.

0 commit comments

Comments
 (0)