A problem about "verify the total derivatives against the finite-difference"

[GreenSoyBeanI]

I met a problem when running the case NACA0012-incompressible.
Here I upload the zip
NACA0012_incompressible_fd_step7.zip
about the case with mesh Generated by script named naca0012_rans.py.
The mesh Generated by naca0012_rans.py is finer than before.
When I set the step size of fd to 1e-5 to 1e-7, I get:
"
Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.patchV'
Analytic Magnitude: 2.314747e-02
Fd Magnitude: 4.256817e-02 (fd:central)
Absolute Error (Jan - Jfd) : 5.369372e-02 *

Relative Error (Jan - Jfd) / Jfd : 1.261358e+00 *

"
even worse
"
Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.shape'
Analytic Magnitude: 1.617682e+01
Fd Magnitude: 1.432387e+02 (fd:central)
Absolute Error (Jan - Jfd) : 1.520629e+02 *

Relative Error (Jan - Jfd) / Jfd : 1.061604e+00 *

"
Could you advise me what I should do to solve this or whether there is something wrong with this case

[101709]

I remember that the two-dimensional example cannot be used to test the derivatives of FFD using the finite difference method. Because simply moving a FFD point would violate the symmetrical boundary conditions on both sides. I think you could try manually perturbing a pair of symmetrical FFD points to calculate a flow field, and then calculate the gradient using the finite difference method once. Compare this gradient value with the result obtained by reverse automatic differentiation of these two FFD points (sum these two automatic differentiation results), and see if they are very close. If you have the comparison results, please feel free to reply to me.

[friedenhe]

@GreenSoyBeanI you should not use such a small step in finite-difference. Try to use 1e-3 or 1e-4. @101709 Yes, we can use the 2D case for verifying the totals, we have fixed the issues in v4, check the latest NACA0012 tutorials

[GreenSoyBeanI]

First I fixed some issues in my runScript.py and FFD.py because the span changed between the mesh generated by naca0012_rans.py and genAirFoilMesh.py

Then I compared the results of different mesh generated scripts, first layer height and different fd steps.
Here are the Details:

I tried 1e-4 as step in fd with mesh generated by naca0012_rans.py(first layer: 1e-6) but I still get:

  Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.patchV'
    Analytic Magnitude: 2.764655e-02
          Fd Magnitude: 1.359464e-01 (fd:central)
    Absolute Error (Jan - Jfd) : 1.147599e-01 *

    Relative Error (Jan - Jfd) / Jfd : 8.441555e-01 *

  Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.shape'
    Analytic Magnitude: 3.423549e-01
          Fd Magnitude: 1.886354e+00 (fd:central)
    Absolute Error (Jan - Jfd) : 1.585726e+00 *

    Relative Error (Jan - Jfd) / Jfd : 8.406298e-01 *

  Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.patchV'
    Analytic Magnitude: 1.514406e+00
          Fd Magnitude: 7.432455e-01 (fd:central)
    Absolute Error (Jan - Jfd) : 7.887462e-01 *

    Relative Error (Jan - Jfd) / Jfd : 1.061219e+00 *

  Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.shape'
    Analytic Magnitude: 5.078617e+01
          Fd Magnitude: 4.637052e+01 (fd:central)
    Absolute Error (Jan - Jfd) : 1.840503e+01 *

    Relative Error (Jan - Jfd) / Jfd : 3.969123e-01 *

---

---

After that I tried using mesh generated by genAirFoilMesh.py with:

NpExtrude = 73
yWall = 1e-5

Here I upload the script:
genAirFoilMesh.py

With this mesh I get(fd step=1e-4):

  Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.patchV'
    Analytic Magnitude: 3.268527e-03
          Fd Magnitude: 3.128078e-03 (fd:central)
    Absolute Error (Jan - Jfd) : 1.404487e-04 *

    Relative Error (Jan - Jfd) / Jfd : 4.489937e-02 *

  Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.shape'
    Analytic Magnitude: 5.347828e-02
          Fd Magnitude: 5.490314e-02 (fd:central)
    Absolute Error (Jan - Jfd) : 1.647356e-03 *

    Relative Error (Jan - Jfd) / Jfd : 3.000477e-02 *

Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.patchV'
   Analytic Magnitude: 1.481560e-01
         Fd Magnitude: 1.371752e-01 (fd:central)
   Absolute Error (Jan - Jfd) : 1.229013e-02 *

   Relative Error (Jan - Jfd) / Jfd : 8.959442e-02 *

  Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.shape'
    Analytic Magnitude: 4.968608e+00
          Fd Magnitude: 4.976619e+00 (fd:central)
    Absolute Error (Jan - Jfd) : 2.617743e-02 *

    Relative Error (Jan - Jfd) / Jfd : 5.260084e-03 *

---

Then I set fd step=1e-3, I get:

  Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.patchV'
    Analytic Magnitude: 3.268527e-03
          Fd Magnitude: 3.286287e-03 (fd:central)
    Absolute Error (Jan - Jfd) : 1.827094e-05 *

    Relative Error (Jan - Jfd) / Jfd : 5.559751e-03 *

Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.shape'
   Analytic Magnitude: 5.347828e-02
         Fd Magnitude: 5.486445e-02 (fd:central)
   Absolute Error (Jan - Jfd) : 1.786145e-03 *

   Relative Error (Jan - Jfd) / Jfd : 3.255560e-02 *

  Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.patchV'
    Analytic Magnitude: 1.481560e-01
          Fd Magnitude: 1.469854e-01 (fd:central)
    Absolute Error (Jan - Jfd) : 1.174869e-03 *

    Relative Error (Jan - Jfd) / Jfd : 7.993102e-03 *

  Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.shape'
    Analytic Magnitude: 4.968608e+00
          Fd Magnitude: 4.992008e+00 (fd:central)
    Absolute Error (Jan - Jfd) : 2.926043e-02 *

    Relative Error (Jan - Jfd) / Jfd : 5.861454e-03 *

---

---

And I also tested with mesh generated by naca0012_rans.py and changed first layer to 1e-5
fd step=1e-4:

  Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.patchV'
    Analytic Magnitude: 2.754161e-02
          Fd Magnitude: 3.837686e-02 (fd:central)
    Absolute Error (Jan - Jfd) : 1.118863e-02 *

    Relative Error (Jan - Jfd) / Jfd : 2.915463e-01 *

  Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.shape'
    Analytic Magnitude: 3.388396e-01
          Fd Magnitude: 3.608734e-01 (fd:central)
    Absolute Error (Jan - Jfd) : 3.992129e-02 *

    Relative Error (Jan - Jfd) / Jfd : 1.106241e-01 *

  Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.patchV'
    Analytic Magnitude: 1.513179e+00
          Fd Magnitude: 1.468067e+00 (fd:central)
    Absolute Error (Jan - Jfd) : 6.977280e-02 *

    Relative Error (Jan - Jfd) / Jfd : 4.752699e-02 *

  Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.shape'
    Analytic Magnitude: 5.060980e+01
          Fd Magnitude: 4.936352e+01 (fd:central)
    Absolute Error (Jan - Jfd) : 1.946099e+00 *

    Relative Error (Jan - Jfd) / Jfd : 3.942383e-02 *

---

fd step=1e-3:

  Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.patchV'
    Analytic Magnitude: 2.754161e-02
          Fd Magnitude: 2.648448e-02 (fd:central)
    Absolute Error (Jan - Jfd) : 1.067879e-03 *

    Relative Error (Jan - Jfd) / Jfd : 4.032093e-02 *

  Full Model: 'scenario1.aero_post.functionals.CD' wrt 'dvs.shape'
    Analytic Magnitude: 3.388396e-01
          Fd Magnitude: 3.414676e-01 (fd:central)
    Absolute Error (Jan - Jfd) : 3.641686e-03 *

    Relative Error (Jan - Jfd) / Jfd : 1.066481e-02 *

  Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.patchV'
    Analytic Magnitude: 1.513179e+00
          Fd Magnitude: 1.412021e+00 (fd:central)
    Absolute Error (Jan - Jfd) : 1.203778e-01 *

    Relative Error (Jan - Jfd) / Jfd : 8.525214e-02 *

  Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.shape'
    Analytic Magnitude: 5.060980e+01
          Fd Magnitude: 5.033169e+01 (fd:central)
    Absolute Error (Jan - Jfd) : 3.691103e-01 *

    Relative Error (Jan - Jfd) / Jfd : 7.333557e-03 *

[101709]

I also would like to ask you, which file controls these output messages? I can't find this file. Also, I would like to ask, is the "check_totals" here obtained using the forward automatic differentiation(Jfor) to get the derivatives? If I want to compare the derivatives obtained through reverse automatic differentiation(maybe Jrev) with those from FD, how should it be set up? @friedenhe

Full Model: 'scenario1.aero_post.functionals.CL' wrt 'dvs.patchV'
    Analytic Magnitude: 1.345505e-01
          Fd Magnitude: 1.342772e-01 (fd:central)
    Absolute Error (Jan - Jfd) : 2.751293e-04 *

    Relative Error (Jan - Jfd) / Jfd : 2.048966e-03 *

    MPI Rank 7

    Raw Analytic Derivative (Jfor)
[[0.10070388 0.0892332 ]]

    Raw FD Derivative (Jfd)
[[0.10047855 0.08907534]]

Although there are 8 independent variables, shouldn't the gradients of the upper and lower control points at the leading and trailing edges be different? How can one obtain the output of 10 control points?

[GreenSoyBeanI]

"check_totals" is used to verify the total derivatives against the finite-difference and "total derivatives" here means derivatives solved by adjoint against the finite-difference.

[GreenSoyBeanI]

problem.py
Here you can find "check_totals". The script is from https://github.com/OpenMDAO/OpenMDAO/blob/master/openmdao/core/problem.py

[GreenSoyBeanI]

You can change AD mode in daoptions.
self.useAD = {"mode": "reverse", "dvName": "None", "seedIndex": -9999}(forward, reverse, or fd)

[101709]

I did set "reverse" in DAoption, but the output information is still "Jfor". I couldn't find which code correspond to these output messages in your link.

[friedenhe]

@101709 Jfor is the adjoint derivative.

[friedenhe]

There are 8 independent shape vars because the two FFD points at the leading edge are linked. Same for the trailing edge. You can see the derivs for all outputs if you set compact_print to False.

[friedenhe]

@GreenSoyBeanI . Are your flow simulations and adjoint computation converge well? I suspect the inaccurate gradients are caused by flow convergence

[GreenSoyBeanI]

Yes, they were. Here are the details about these with fd step = 1e-3

1. Mesh generated by naca0012_rans.py

• adjoint

Main iteration 0 KSP Residual norm 8.498181774416e-01 59.65 s. 
Main iteration 100 KSP Residual norm 9.003637706216e-02 66.20 s. 
Main iteration 200 KSP Residual norm 5.409493588185e-02 73.42 s. 
Main iteration 300 KSP Residual norm 9.418224225105e-04 81.54 s. 
Main iteration 400 KSP Residual norm 4.735251226389e-05 90.56 s. 
Main iteration 489 KSP Residual norm 8.358440140764e-07 99.33 s 
**Completed**! Total iterations: 489. PetscConvergedReason: 2. 99.33 s
Residual tolerance satisfied, solution finished!

• flow simulations

Time = 2100

U0 initRes: 1.737240205546118e-09 finalRes: 1.524262771590848e-10 nIters: 6
U1 initRes: 1.158883346456496e-08 finalRes: 9.763346264993826e-10 nIters: 6
U2 initRes: 4.778473640590832e-08 finalRes: 3.443630701191317e-09 nIters: 6
p initRes: 6.703359121173006e-09 finalRes: 6.001478572926965e-10 nIters: 8
Time step continuity errors : sum local = 9.738436475846661e-12
                                 global = 5.854918268255116e-12
                             cumulative = 6.507723486389563e-05
nuTilda initRes: 2.13051514094206e-09 finalRes: 1.646344439720695e-10 nIters: 3
CD: 0.1401356007129693 final: 0.1401356007129693
CL: 5.457110476492623 final: 5.457110476492623
yPlus min: 0.01701305587047077 max: 0.4574593763993574 mean: 0.1571875862690572
ExecutionTime = 33.43 s  ClockTime = 40 s

Time = 2133
Minimal residual 9.987543446751198e-09 satisfied the prescribed tolerance 1e-08

CD: 0.1401353276866747 final: 0.1401353276866747
CL: 5.457128139602744 final: 5.457128139602744
End

2. Mesh generated by genAirFoilMesh.py

• adjoint

Main iteration 0 KSP Residual norm 1.208443098869e-01 24.71 s. 
Main iteration 100 KSP Residual norm 8.479932795126e-03 26.79 s. 
Main iteration 200 KSP Residual norm 2.841032318410e-04 29.11 s. 
Main iteration 300 KSP Residual norm 1.353142990295e-07 31.70 s. 
Main iteration 303 KSP Residual norm 1.180738828143e-07 31.79 s 
**Completed**! Total iterations: 303. PetscConvergedReason: 2. 31.79 s
Residual tolerance satisfied, solution finished!

• flow simulations

Time = 1100

U0 initRes: 4.901980151756218e-10 finalRes: 4.219496268319949e-11 nIters: 6
U1 initRes: 1.112001604495797e-08 finalRes: 8.958040767549052e-10 nIters: 6
U2 initRes: 6.823851259346014e-08 finalRes: 5.285083723929317e-09 nIters: 6
p initRes: 2.294972012218041e-09 finalRes: 1.8088910494698e-10 nIters: 5
Time step continuity errors : sum local = 7.266900428440341e-12
                                 global = -1.503181188645554e-12
                             cumulative = 0.000374844281875231
nuTilda initRes: 6.14465271918078e-10 finalRes: 6.140968216927922e-11 nIters: 3
CD: 0.0156494823467275 final: 0.0156494823467275
CL: 0.5344916784660342 final: 0.5344916784660342
yPlus min: 0.03016871763857822 max: 0.4416271747863795 mean: 0.1645367687517354
ExecutionTime = 15.12 s  ClockTime = 31 s

Time = 1110
Minimal residual 9.934966875722544e-09 satisfied the prescribed tolerance 1e-08

CD: 0.01564948069921607 final: 0.01564948069921607
CL: 0.5344917770403209 final: 0.5344917770403209
End

[GreenSoyBeanI]

And the case that I said I got inaccurate gradients( mesh generated by naca0012_rans.py with first layer: 1e-6 ) converged well too.

• adjoint

Main iteration 0 KSP Residual norm 8.247923403571e+00 56.85 s. 
Main iteration 100 KSP Residual norm 1.179542133523e-01 63.17 s. 
Main iteration 200 KSP Residual norm 7.048243523770e-02 70.10 s. 
Main iteration 300 KSP Residual norm 2.483809227686e-03 77.81 s. 
Main iteration 400 KSP Residual norm 7.881172372583e-05 86.31 s. 
Main iteration 469 KSP Residual norm 8.102393605993e-06 92.68 s 
**Completed**! Total iterations: 469. PetscConvergedReason: 2. 92.68000000000001 s
Residual tolerance satisfied, solution finished!

• flow simulations

Time = 1800

U0 initRes: 2.013004400958071e-09 finalRes: 1.787459995852866e-10 nIters: 6
U1 initRes: 1.444901768934881e-08 finalRes: 1.250179320968529e-09 nIters: 6
U2 initRes: 8.664928664583978e-08 finalRes: 6.515387856689404e-09 nIters: 6
p initRes: 1.155287475485416e-08 finalRes: 8.253354388320134e-10 nIters: 10
Time step continuity errors : sum local = 1.458054560009632e-11
                                 global = -8.545194840745447e-12
                             cumulative = 0.0001316503718156142
nuTilda initRes: 4.439969431682813e-09 finalRes: 3.752758740370327e-10 nIters: 3
CD: 0.1405721044343204 final: 0.1405721044343204
CL: 5.461268958511983 final: 5.461268958511983
yPlus min: 0.001743144297372984 max: 0.05844108508623587 mean: 0.0160416790790616
ExecutionTime = 30.02 s  ClockTime = 40 s

Time = 1875
Minimal residual 9.961912502350092e-09 satisfied the prescribed tolerance 1e-08

CD: 0.1405729706713606 final: 0.1405729706713606
CL: 5.461203640593761 final: 5.461203640593761
End