Adding cosine similarity as a metric to monitor force errors

[wiederm]

Description
Currently, during training, we monitor the force prediction errors using the following metrics:

• L1 Norm: L1_norm(F_i^{true} - F_i^{pred}) – visualized in the error histograms.
• L2 Norm: L2_norm(F_i^{true} - F_i^{pred}) – visualized in the RMSE/MAE plots.

I propose adding cosine similarity as an additional metric to monitor the alignment between predicted and true force vectors.

Proposed Addition
Cosine Similarity: Compute and monitor the cosine similarity between F_i^{pred} and F_i^{true} during training.
This metric should be plotted alongside existing error histograms and RMSE/MAE plots.

Rationale
Directional Accuracy: While L1 and L2 norms provide information about the magnitude of errors, they do not convey how well the predicted forces align in direction with the true forces. Cosine similarity specifically measures the orientation of the vectors, indicating whether the predictions are directionally accurate. By monitoring cosine similarity, we can discern whether the model's errors are primarily due to incorrect magnitudes, incorrect directions, or a combination of both.