Question about sin activation function

[camillae00]

Hi, I have a question regarding using sin activation function.

I am typically working with inverse problems that involves using measurement data to infer unknown parameters in a set of differential equations that (presumably) describes the system in question. The problems I work with typically have some commonalities:
A) The differential equation(s) are simplified descriptions of the system in question, e.g. a lumped description of a system
B) The measured data is usually periodic
C) The measurements have noise (it varies to which degree, but it can be a lot)

I have experience that for these kind of problems using DeepXDE, it usually works significantly better (in terms of loss) to use the "sin" activation function rather than "tanh". However, when I look at the resulting functions that they predict it seems that the "sin" overfits the function, and from a physical/mathematical viewpoint it does not look very "correct" (see images below).

Here, the dots are the measurement points and the dashed lines are the predictions. One is done using tanh and one with sin

I am wondering why these two activation functions give such different results? And how should interpret the "sin" solution. It obviously fits the measurement data quite well, but it doesn't look "physical" to me. What is the significance of choosing different activation functions?

[lululxvi]

What I can say is that activation function is definitely important. Which one to use depends on your problem.