[Question] Guidance on max_steps setting and retrieving validation loss history

[Charles-T]

Hello,

I noticed that max_steps is often set to 1000 in many examples and default settings.

I am currently working with a dataset containing 20,000 time series, each with a length of 400 timestamps. I have a few questions regarding the training configuration:

max_steps vs n_epochs: Could you explain why max_steps is preferred over n_epochs in NeuralForecast?
Hyperparameter adjustment: If I set batch_size=1024, given the size of my dataset, the model might not fully traverse the data (or complete sufficient epochs) within 1000 steps. In this scenario, is it recommended to increase max_steps?
Validation Loss: Is there a built-in method or attribute to access the validation loss curve during or after the training process?
Thanks for your help!

[marcopeix]

Hello!

1. We decided to use max_steps instead of epochs a while ago to have consistent training no matter the number of series in your dataset. So, epochs depend on the number of series, batch size and their length, but not training steps. This also helps for training multiple global models, like we do in neuralforecast.

2. Indeed, if you want to ensure the model sees all data, and your dataset is very large, increase the number of steps. We set it to 1000 as a general good starting point.

3. For the validation, you can do something like this to access it and plot it:

mdl = nf.models[0]
train = mdl.train_trajectories  
valid = mdl.valid_trajectories

df_loss = pd.DataFrame(train, columns=['x', 'y'])
plt.plot(df_loss['x'], df_loss['y'])
plt.title("Training Loss") 
plt.show()

df_loss = pd.DataFrame(valid, columns=['x', 'y'])
plt.plot(df_loss['x'], df_loss['y']) 
plt.title("Validation Loss")
plt.show()

Note that nf.models[0] accesses the first model. So adjust the index if you have a second or third model.

Let me know if it helps!

[Charles-T]

Thank you for the clear explanation!
Also, the code for accessing train_trajectories and valid_trajectories is exactly what I needed.
I appreciate your help!