results depend on integration time

[nchoksi4456]

I've been simulating two-planet systems and noticed in some cases that I get totally different evolutions depending on the max time I run for. For example, the following code:

sim.automateSimulationArchive("test1.bin", interval=1*cgs.yr,deletefile=True) 
sim.move_to_com()
max_time = 3e4*cgs.yr  # Run for 3e4 yr 
Noutputs = int(( max_time/dt)) # I've been using dt = 1 yr as a default
times = np.linspace(0,  max_time, Noutputs)
for i,tv in enumerate(times): 
      sim.integrate(tv, exact_finish_time = 0)

produces plot test1 below.

But if I halve the runtime to max_time = 1.5e4*cgs.yr (all else left fixed), I get the drastically different evolution seen in test2. For example, in test1, the two planets collide with each other at ~7e3 yr (you can see the pink curve cuts off at that time), and in test2 one of the planets collides with the star at 1e4 yr.

In case it is relevant, I'm using IAS15 and including damping via a (modified) version of your modify_orbits_forces script.

[dtamayo]

Hi Nick,

This is a good question that a lot of people run into. The issue is that when planets are colliding the evolution is chaotic, which means that any change can lead to completely different outcomes. In this case, the fact that your two simulations stop at different arrays of times, means that differences at machine precision grow exponentially to change the phases and times of collision in the two simulations.

If you want to be able to get the exact same trajectory, this is where SimulationArchives are very helpful (look at the two examples in ipython_examples). The idea is that you run a simulation saving snapshots in a simulation archive (and don't take any output). Then when you want to analyze the simulation, you can get output at arbitrary times, and you're always referencing the same trajectory (since you're loading the nearest snapshot of the exact same chaotic realization each time, and integrating to your output time without there being enough time for chaos to completely change the answer). See Rein & Tamayo 2016 for more details.

Once you start using them you'll never go back :)

[hannorein]

I don't think this is correct. Nick is using exact_finish_time = 0 so the times where he samples the simulation should not matter. To say more, I'd need a working example.

[hannorein]

Here's a simple example. Both runs return exactly the same position as expected. It might be that there's something else in Nicks code which breaks reproducibility.

import rebound
import rebound.data
sim = rebound.Simulation()
rebound.data.add_outer_solar_system(sim)
sim.integrate(1,exact_finish_time=0)
sim.integrate(2,exact_finish_time=0)
"%.20f"%sim.particles[1].x

sim = rebound.Simulation()
rebound.data.add_outer_solar_system(sim)
sim.integrate(2,exact_finish_time=0)
"%.20f"%sim.particles[1].x

[nchoksi4456]

Hi Hanno and Dan. I've uploaded a full working example "test.py" here as requested. One issue is that I am using my own version of modify_orbits_forces -- I've also included the script "df2_full.c" for that here but you'll just have to attach it to REBOUNDx on your machines to run "test.py" (i.e., add to "core.c" and "core.h" in REBOUNDx, etc.). (BTW, the point of this revised modify_orbits_forces is to implement a Chandrasekhar-like dynamical friction based on the local gas properties.)

Thanks so much for your help!

[nchoksi4456]

Hi Hanno and Dan. FYI the "test.py" script I uploaded ~40 minutes ago used the wrong random seed. It should be np.random.seed(0) to get the same initial conditions as I've been using. I updated the file at the link with the correct random seed, but just in case you already downloaded the script already you'll want to make this change. Sorry for the confusion!

[hannorein]

I did a quick test without REBOUNDx. That leads to reproducible results for me, independent on max_time. @dtamayo, can you think of anything that REBOUNDx does to make it non-reproducible?

[dtamayo]

Oops, thanks @hannorein ! I don't see anything obvious in your code Nick. I made a similar test but using the actual modify_orbits_forces routine, and got reproducible results independent of max time, so I don't think it's something intrinsic to REBOUNDx.

My advice would be to start from a similar test case --changing what you have to using a simple toy case with "modify_orbits_forces" with a real 'tau_a', and once you verify that that test case gives you reproducible results independent of tmax, then switch in your new function one step at a time, e.g., switch to your function, but at first comment out everything in the function so that it isn't doing anything, then add in blocks of your function one at a time to isolate which one is causing the problem.

I'm happy to help more with what you find!

[hannorein]

Note that you don't need to run it for very long or need to do any plots. Just compare any coordinate of any particle at any given time. The coordinates should be identical to the last digit. As soon as you see a difference in the last digit, you can stop (the differences will grow exponentially from thereon, as Dan explained).

[nchoksi4456]

Thanks Hanno and Dan, I'll try out these suggestions. Could I ask one more question right now, which might or might not be related? To record output, I was using automateSimulationArchive with a fixed interval = 100 yr. And yet, I noticed that in the 1.5e4 yr run that the output intervals of the simulationArchive change after one of the planets collides with the star. The plot below shows the # of planets vs. time, and you can see that the outputs become more sparse after the collision (before the collision all the outputs are spaced at 100 yr intervals as they should be). Do you know why this happens?

[hannorein]

Is the semi-major axis so large that the timestep is larger than 100yr?

[nchoksi4456]

Thanks Hanno, that's probably it... a(t) is wild.

[hannorein]

Keep us posted! I'm curious to find out what the issue is. Just to make sure: are you using the latest version of REBOUND? Did you make any modifications?

[nchoksi4456]

Will do! And yes, I'm using the latest version of REBOUND, but haven't made any changes to it.

[nchoksi4456]

Here's another diagnostic test I tried which continues to add to my confusion. If I just replace

Noutputs = int(( max_time/dt)) # I've been using dt = 1 yr as a default
times = np.linspace(0,  max_time, Noutputs)
for i,tv in enumerate(times): 
      sim.integrate(tv, exact_finish_time = 0)

with

times = np.arange(0,  max_time, step = 1*yr)
for i,tv in enumerate(times): 
      sim.integrate(tv, exact_finish_time = 0)

The results don't depend on integration time...

[hannorein]

Hm. Strange. Just to make sure: Are the timesteps always shorter than the output interval? If not, you might end up giving sim.integrate() a time in the past (and it will integrate backwards). Maybe add a check in your loop to make sure?

if tv<sim.t:
    raise RuntimeError("requested time is in the past")