Using K-FAC with physics-based losses

[SNMS95]

Hey,

Thank you for the implementation.

From the guide, I saw that I have to register loss functions to be able to use K-FAC.
For my specific case, the loss function is a FEM simulation on the outputs of the network along with some other functions (postprocessing, filtering etc).

Will it be possible to use K-FAC?

[SNMS95]

@james-martens

Hi James,

Is it possible to create a custom loss for KFAC?

[SNMS95]

Hey,

Thank you for the implementation.

From the guide, I saw that I have to register loss functions to be able to use K-FAC. For my specific case, the loss function is a FEM simulation on the outputs of the network along with some other functions (postprocessing, filtering etc).

Will it be possible to use K-FAC?

Hi,

Has there been any update on it?

[james-martens]

Hi,

Sorry for the late reply.

The K-FAC method is based on the generalized Gauss-Newton matrix (GGN), which required a convex loss (as a function of the network's output). If your loss is convex then you could implement your loss function class and registration functions, similar to the ones that exist already in the package. We won't be accepting any contributions, but you're free to do this in your own fork or via careful importing and sublassing.

Another possibility here is that you might want to view parts of your complicated loss function as being formally part of the network function. If the remainder of the loss function is convex, or even better has the form of an already supported loss function, then that could work too.

James