use a list of output layer in the PFNN

[enigne]

Hi,

I'm currently using deepxde to do ice sheet modeling , TensorFlow2 backend.
I would like to ask if it is possible to make deepxde.nn.PFNN to be able to handle output layers defined by a list, instead of by an integer.
For example,
deepxde.nn.PFNN([2, [20,20,20], [20,20,20], [2,2,1]],"tanh", "Glorot uniform")
which is a parallel net with 2 inputs, and 5 (in total) outputs, but clustered into 2,2,1. In this case, we will get 3 parallel nets, but 5 different output.

[haison19952013]

• I think you should modify line in the class PFNN(NN) as follows:

class PFNN(NN):
    """Parallel fully-connected neural network that uses independent sub-networks for
    each network output.

    Args:
        layer_sizes: A nested list to define the architecture of the neural network (how
            the layers are connected). If `layer_sizes[i]` is int, it represent one
            layer shared by all the outputs; if `layer_sizes[i]` is list, it represent
            `len(layer_sizes[i])` sub-layers, each of which exclusively used by one
            output. Note that `len(layer_sizes[i])` should equal to the number of
            outputs. Every number specify the number of neurons of that layer.
    """

    def __init__(
        self, layer_sizes, activation, kernel_initializer, regularization=None
    ):
        super().__init__()
        activation = activations.get(activation)
        initializer = initializers.get(kernel_initializer)
        self.regularizer = regularizers.get(regularization)
       ######### FIX HERE #########
        if isinstance(layer_sizes[-1], (list, tuple)) and len(layer_sizes[-1])>1:
             n_output = len(layer_sizes[-1])
        else:
             n_output = layer_sizes[-1]
       ######### END #########
        self.denses = []
        # hidden layers
        for i in range(1, len(layer_sizes) - 1):
            prev_layer_size = layer_sizes[i - 1]
            curr_layer_size = layer_sizes[i]
            # Non-Shared layers
            if isinstance(curr_layer_size, (list, tuple)):
                if len(curr_layer_size) != n_output:
                    raise ValueError(
                        "number of sub-layers should equal number of network outputs"
                    )
                # e.g. [8, 8, 8] -> [16, 16, 16] or 64 -> [8, 8, 8]
                self.denses.append(
                    [
                        tf.keras.layers.Dense(
                            units,
                            activation=activation,
                            kernel_initializer=initializer,
                            kernel_regularizer=self.regularizer,
                        )
                        for units in curr_layer_size
                    ]
                )
            # Shared layers
            else:  # e.g. 64 -> 64
                if not isinstance(prev_layer_size, int):
                    raise ValueError(
                        "cannot rejoin parallel subnetworks after splitting"
                    )
                self.denses.append(
                    tf.keras.layers.Dense(
                        curr_layer_size,
                        activation=activation,
                        kernel_initializer=initializer,
                        kernel_regularizer=self.regularizer,
                    )
                )

        # output layers
        ######### FIX HERE#########
        if isinstance(layer_sizes[-2], (list, tuple)):  # e.g. [3, 3, 3] -> 3
            self.denses.append(
                [
                    tf.keras.layers.Dense(
                        layer_sizes[-1][i],
                        kernel_initializer=initializer,
                        kernel_regularizer=self.regularizer,
                    )
                    for i in range(len(layer_sizes[-1]))
                ]
            )
        ######### END #########
        else:
            self.denses.append(
                tf.keras.layers.Dense(
                    n_output,
                    kernel_initializer=initializer,
                    kernel_regularizer=self.regularizer,
                )
            )

• The idea is to change the defined output size in line 127. Therefore, the number of outputs of each NN will follow your settings. Plz check again if there is any mistake in the code. Hope you understand the idea.