Using multiple system trajectories as training data for NODE with exogenous inputs

[friegben]

Hello everyone,

I would like to ask you the following question:

Within the example provided here https://colab.research.google.com/github/pnnl/neuromancer/blob/master/examples/ODEs/Part_4_nonauto_NODE.ipynb#scrollTo=56b94c2f, it is stated that

"Starting from a given initial condition x0 , the next state of the system xk+1 is obtained by feeding the current state xk and current input uk into the model S . In system identification, the loss L is evaluated by comparing the trajectory generated by the model with the training trajectory. The process can be repeated for multiple trajectories to improve the generalization of the model."

My questions are related to the highlighted part: Is there a way to train the NODE based on multiple system trajectories as suggested by the highlighted part?

To clarify: I am trying to use neuromancer's NODE capabilities to train a neural net to identify the dynamics of a known system. From this known system, I generate the training data. However, I would like to generate multiple system trajectories from that known system i.e. system trajectories that differ in their initial condition (and possibly exogenous inputs). This way, I am trying to get the neural net to learn the general system dynamics and not just the dynamics from one initial condition. Is there an example I am missing that already demonstrates this? If not, how would I achieve my goal of including multiple system trajectories that each differ in their initial condition?

How do I repeat for multiple trajectories to improve the generalization of the model?

Thank you already - neuromancer has been very helpful for all kinds of tasks I need to accomplish :)

[friegben]

I spent some time thinking about this during the weekend and I think I found the solution:
Lets say I have 10 simulation runs using my known system (using different initial conditions for each run) and each run includes 52 timesteps, i.e. 520 in total.
If I choose nbatch=10 (1 batch for 1 simulation run) and nsteps=52 (prediction horizon in the objective function), each simulation run (i.e. system trajectory) is independently evaluated in the objective function. This way, no data point from one trajectory slips into the evaluation of a different trajectory in the objective function. This is probably obvious for most people, but I am quite new to training neural nets and for me it took a long time to get right :)

[drgona]

Hi @friegben that's right you can simulate multiple trajectories from different initial conditions and treat each as a single batch. You can also cut one long trajectory into multiple smaller trajectories to increase the batch number.