Skip to content

Remove buggy synchronization at end of steps #5820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

dpgrote
Copy link
Member

@dpgrote dpgrote commented Apr 8, 2025

This PR does cleanup, removing a potential bug that happens when the velocity is synchronized with position in the final diagnostic. The particle boundary conditions had not been applied so the field gather could be out of bounds.

Further cleanup is done, renaming Synchronize to SynchronizeVelocityWithPosition, and having it check is_synchronized, since the synchronization may have already been done.

Note that the factor 1e-3 in the code change

bool const end_of_step_loop = (step == numsteps_max - 1) || (cur_time + dt[0] >= stop_time - 1.e-3*dt[0]);

is a somewhat arbitrary factor to allow for round off error in the time. If the cur_time + dt[0] is almost but not quite equal to stop_time, then that's close enough. Comment copied here from #5820 (comment) for better visibility.

Rename Synchronize to SynchronizeVelocityWithPosition.
SynchronizeVelocityWithPosition checks is_synchronized.
@dpgrote dpgrote added bug Something isn't working cleaning Clean code, improve readability labels Apr 8, 2025
@EZoni EZoni requested review from RemiLehe and EZoni April 15, 2025 23:47
@RemiLehe
Copy link
Member

@dpgrote Thanks for this PR! Would you be able to merge development into this PR and update the checksums?

*m_fields.get(FieldType::Bfield_aux, Direction{1}, lev),
*m_fields.get(FieldType::Bfield_aux, Direction{2}, lev)
);
if (!is_synchronized) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be one of those examples where making it clear that is_synchronized is a member variable of the WarpX class, through the prefix m_, would help not get lost while reading the code. For a minute I was trying to understand where is_synchronized is defined in this function. We could rename it according to the WarpX style rules in a separate PR.

@EZoni
Copy link
Member

EZoni commented Apr 24, 2025

Two big PRs are being merged as we speak:

Since those PRs introduce new CI tests, I would recommend that we merge development here, after those PRs are merged, before resetting the checksums in this PR.

@aeriforme
Copy link
Member

aeriforme commented May 1, 2025

I could use this fix after #3630 was merged.
I have some simulations (that use that PR) where the last time step fails, not matter which step it is.
I merged this branch and it works fine.

@EZoni EZoni added the bug: affects latest release Bug also exists in latest release version label May 1, 2025
@EZoni
Copy link
Member

EZoni commented May 1, 2025

After the last merge of development the checksums need a new update. Once that's done, I think this PR would be ready to approve and merge. Let's aim at having this merged before the hackathon of May 12-14.

@EZoni EZoni mentioned this pull request May 6, 2025
3 tasks
@EZoni
Copy link
Member

EZoni commented May 6, 2025

Thanks for all the updates, @dpgrote. I think test_3d_effective_potential_electrostatic_picmi is the only remaining test that needs a checksum update. It could be nice to merge this before the release update in #5865.

@EZoni EZoni self-requested a review May 6, 2025 15:33
@@ -351,12 +337,13 @@ WarpX::Evolve (int numsteps)
}

bool const do_diagnostic = (multi_diags->DoComputeAndPack(step) || reduced_diags->DoDiags(step));
bool const end_of_step_loop = (step == numsteps_max - 1) || (cur_time + dt[0] >= stop_time - 1.e-3*dt[0]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a question to clarify, where does the 1.e-3 factor come from here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a somewhat arbitrary factor to allow for round off error in the time. If the cur_time + dt is almost but not quite equal to stop_time, then that's close enough.

assert Ep_f < 0.7 * Ep_i # Check that potential energy changes significantly
assert abs((Ek_i + Ep_i) - (Ek_f + Ep_f)) < 0.003 * (
assert abs((Ek_i + Ep_i) - (Ek_f + Ep_f)) < 0.0032 * (
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why the error is worse here than before. Note that this has warpx.synchronize_velocity_for_diagnostics turned on. With it off, the error is a factor of several worse.

@EZoni EZoni merged commit 444bd7e into BLAST-WarpX:development May 7, 2025
37 checks passed
@dpgrote dpgrote deleted the remove_end_of_step_synchronization branch May 7, 2025 22:47
RemiLehe pushed a commit that referenced this pull request May 9, 2025
Weekly update to latest AMReX, pyAMReX, PICSAR:
```
./Tools/Release/updateAMReX.py
./Tools/Release/updatepyAMReX.py
./Tools/Release/updatePICSAR.py
```

WarpX release 25.05:
```
./Tools/Release/newVersion.sh
```

To-do:
- [x] Merge AMReX-Codes/pyamrex#442
- [x] Merge #5820
- [x] Mark as ready for review and run CI
atmyers pushed a commit to atmyers/WarpX that referenced this pull request Jul 3, 2025
This PR does cleanup, removing a potential bug that happens when the
velocity is synchronized with position in the final diagnostic. The
particle boundary conditions had not been applied so the field gather
could be out of bounds.

Further cleanup is done, renaming Synchronize to
SynchronizeVelocityWithPosition, and having it check is_synchronized,
since the synchronization may have already been done.

Note that the factor `1e-3` in the code change
```cpp
bool const end_of_step_loop = (step == numsteps_max - 1) || (cur_time + dt[0] >= stop_time - 1.e-3*dt[0]);
```
is a somewhat arbitrary factor to allow for round off error in the time.
If the `cur_time + dt[0]` is almost but not quite equal to `stop_time`,
then that's close enough. Comment copied here from
BLAST-WarpX#5820 (comment)
for better visibility.
atmyers pushed a commit to atmyers/WarpX that referenced this pull request Jul 3, 2025
Weekly update to latest AMReX, pyAMReX, PICSAR:
```
./Tools/Release/updateAMReX.py
./Tools/Release/updatepyAMReX.py
./Tools/Release/updatePICSAR.py
```

WarpX release 25.05:
```
./Tools/Release/newVersion.sh
```

To-do:
- [x] Merge AMReX-Codes/pyamrex#442
- [x] Merge BLAST-WarpX#5820
- [x] Mark as ready for review and run CI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug: affects latest release Bug also exists in latest release version bug Something isn't working cleaning Clean code, improve readability
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move velocity and position synchronization to appropriate place in PIC loop
4 participants