Cannot get losses.mean_l2_relative_error to work with paddle

[Saransh-cpp]

I have been experimenting with multiple backends but for some reason, I cannot get the mean_l2_relative_error loss to work with the paddle backend -

import deepxde as dde
import numpy as np
import paddle


# all the lines below error out
dde.losses.mean_l2_relative_error(np.array([[1, 1]]), np.array([[2, 2]]))
dde.losses.mean_l2_relative_error(1, 2)
dde.losses.mean_l2_relative_error(paddle.to_tensor(1.0), paddle.to_tensor(2.0)))

Error -

ValueError                                Traceback (most recent call last)
Cell In[6], line 1
----> 1 dde.losses.mean_l2_relative_error(np.array([[1, 1]]), np.array([[2, 2]]))

File ~/Code/SciML/deepxde/deepxde/losses.py:27, in mean_l2_relative_error(y_true, y_pred)
     26 def mean_l2_relative_error(y_true, y_pred):
---> 27     return bkd.reduce_mean(bkd.norm(y_true - y_pred, axis=1) / bkd.norm(y_true, axis=1))

File ~/Code/SciML/deepxde/deepxde/backend/paddle/tensor.py:176, in norm(x, ord, axis, keepdims)
    175 def norm(x, ord=None, axis=None, keepdims=False):
--> 176     return paddle.linalg.norm(x, p=ord, axis=axis, keepdim=keepdims)

File ~/Library/Python/3.9/lib/python/site-packages/paddle/tensor/linalg.py:593, in norm(x, p, axis, keepdim, name)
    584         return vector_norm(
    585             x,
    586             axis=axis,
   (...)
    590             name=name,
    591         )
    592     else:
--> 593         raise ValueError(
    594             "unspport p for p-order vector norm. except float, found {}".format(
    595                 p
    596             )
    597         )
    598 # calculate matrix norm, where axis is list with two integers
    599 elif isinstance(axis, list) and len(axis) == 2:

ValueError: unspport p for p-order vector norm. except float, found None

Full stack trace -

Full trace

x = Tensor(shape=[2, 1], dtype=float64, place=Place(cpu), stop_gradient=True,
       [[1.],
        [1.]]), p = None, axis = 1, keepdim = False, name = None

    def norm(x, p='fro', axis=None, keepdim=False, name=None):
        """
    
        Returns the matrix norm (Frobenius) or vector norm (the 1-norm, the Euclidean
        or 2-norm, and in general the p-norm for p > 0) of a given tensor.
    
        Note:
            This norm API is different from `numpy.linalg.norm`.
            This api supports high-order input tensors (rank >= 3), and certain axis need to be pointed out to calculate the norm.
            But `numpy.linalg.norm` only supports 1-D vector or 2-D matrix as input tensor.
            For p-order matrix norm, this api actually treats matrix as a flattened vector to calculate the vector norm, NOT REAL MATRIX NORM.
    
        Args:
            x (Tensor): The input tensor could be N-D tensor, and the input data
                type could be float32 or float64.
            p (float|string, optional): Order of the norm. Supported values are `fro`, `0`, `1`, `2`,
                `inf`, `-inf` and any positive real number yielding the corresponding p-norm. Not supported: ord < 0 and nuclear norm.
                Default value is `fro`.
            axis (int|list|tuple, optional): The axis on which to apply norm operation. If axis is int
                or list(int)/tuple(int)  with only one element, the vector norm is computed over the axis.
                If `axis < 0`, the dimension to norm operation is rank(input) + axis.
                If axis is a list(int)/tuple(int) with two elements, the matrix norm is computed over the axis.
                Default value is `None`.
            keepdim (bool, optional): Whether to reserve the reduced dimension in the
                output Tensor. The result tensor will have fewer dimension
                than the :attr:`input` unless :attr:`keepdim` is true, default
                value is False.
            name (str, optional): The default value is None. Normally there is no need for
                user to set this property. For more information, please refer to :ref:`api_guide_Name`.
    
        Returns:
            Tensor: results of norm operation on the specified axis of input tensor,
            it's data type is the same as input's Tensor.
    
        Examples:
            .. code-block:: python
    
                import paddle
                x = paddle.arange(24, dtype="float32").reshape([2, 3, 4]) - 12
                # x: Tensor(shape=[2, 3, 4], dtype=float32, place=Place(cpu), stop_gradient=True,
                #          [[[-12., -11., -10., -9. ],
                #            [-8. , -7. , -6. , -5. ],
                #            [-4. , -3. , -2. , -1. ]],
    
                #           [[ 0. ,  1. ,  2. ,  3. ],
                #            [ 4. ,  5. ,  6. ,  7. ],
                #            [ 8. ,  9. ,  10.,  11.]]])
    
                # compute frobenius norm along last two dimensions.
                out_fro = paddle.linalg.norm(x, p='fro', axis=[0,1])
                # out_fro: Tensor(shape=[4], dtype=float32, place=Place(cpu), stop_gradient=True,
                #                 [17.43559647, 16.91153526, 16.73320007, 16.91153526])
    
                # compute 2-order vector norm along last dimension.
                out_pnorm = paddle.linalg.norm(x, p=2, axis=-1)
                # out_pnorm: Tensor(shape=[2, 3], dtype=float32, place=Place(cpu), stop_gradient=True,
                #                [[21.11871147, 13.19090557, 5.47722578 ],
                #                 [3.74165750 , 11.22497177, 19.13112640]])
    
                # compute 2-order  norm along [0,1] dimension.
                out_pnorm = paddle.linalg.norm(x, p=2, axis=[0,1])
                # out_pnorm: Tensor(shape=[4], dtype=float32, place=Place(cpu), stop_gradient=True,
                #                  [17.43559647, 16.91153526, 16.73320007, 16.91153526])
    
                # compute inf-order  norm
                out_pnorm = paddle.linalg.norm(x, p=float("inf"))
                # out_pnorm  = Tensor(shape=[1], dtype=float32, place=Place(cpu), stop_gradient=True,
                #                    [12.])
    
                out_pnorm = paddle.linalg.norm(x, p=float("inf"), axis=0)
                # out_pnorm: Tensor(shape=[3, 4], dtype=float32, place=Place(cpu), stop_gradient=True,
                #                 [[12., 11., 10., 9. ],
                #                  [8. , 7. , 6. , 7. ],
                #                  [8. , 9. , 10., 11.]])
    
                # compute -inf-order  norm
                out_pnorm = paddle.linalg.norm(x, p=-float("inf"))
                # out_pnorm: Tensor(shape=[1], dtype=float32, place=Place(cpu), stop_gradient=True,
                #                  [0.])
    
                out_pnorm = paddle.linalg.norm(x, p=-float("inf"), axis=0)
                # out_pnorm: Tensor(shape=[3, 4], dtype=float32, place=Place(cpu), stop_gradient=True,
                #                  [[0., 1., 2., 3.],
                #                  [4., 5., 6., 5.],
                #                  [4., 3., 2., 1.]])
        """
    
        def frobenius_norm(input, dim=None, keepdim=False, name=None):
            """
            The frobenius norm OP is to calculate the frobenius norm of certain two dimensions of Tensor `input`.
            Args:
              input (Variable): Tensor, data type float32, float64.
              dim (list, optional): None for last two dimensions.
              keepdim (bool, optional): Whether keep the dimensions as the `input`, Default False.
            """
            if dim is not None and not (isinstance(dim, list) and len(dim) == 2):
                raise ValueError(
                    "The dim of frobenius norm op should be None or two elements list!"
                )
    
            if in_dygraph_mode():
                if dim is None:
                    return _C_ops.frobenius_norm(input, [], keepdim, True)
                return _C_ops.frobenius_norm(input, dim, keepdim, False)
            else:
                attrs = {'dim': dim, 'keep_dim': keepdim, 'reduce_all': False}
                if dim is None:
                    attrs['reduce_all'] = True
                check_variable_and_dtype(
                    input, 'input', ['float32', 'float64'], 'frobenius_norm'
                )
    
                helper = LayerHelper('frobenius_norm', **locals())
                out = helper.create_variable_for_type_inference(
                    dtype=helper.input_dtype()
                )
    
                helper.append_op(
                    type='frobenius_norm',
                    inputs={'X': input},
                    outputs={'Out': out},
                    attrs=attrs,
                )
                return out
    
        def vector_norm(
            input, porder=None, axis=None, keepdim=False, asvector=False, name=None
        ):
            """
            Calculate the p-order vector norm for certain  dimension of Tensor `input`.
            Args:
              input (Variable): Tensor, data type float32, float64.
              porder (float, optional): None for porder=2.0.
              axis (int, optional): None for last dimension.
              keepdim (bool, optional): Whether keep the dimensions as the `input`, Default False.
            """
            if in_dygraph_mode():
                if axis is None:
                    axis = -1
                return _C_ops.p_norm(input, porder, axis, 1e-12, keepdim, asvector)
            else:
                if porder is not None:
                    check_type(porder, 'porder', (float, int), 'p_norm')
                if axis is not None:
                    check_type(axis, 'axis', (int), 'p_norm')
                check_variable_and_dtype(
                    input,
                    'input',
                    ['float16', 'uint16', 'float32', 'float64'],
                    'p_norm',
                )
    
                attrs = {
                    'axis': axis if axis is not None else -1,
                    'porder': float(porder) if porder is not None else 2.0,
                    'keepdim': keepdim,
                    'asvector': asvector,
                    'epsilon': 1e-12,
                }
                helper = LayerHelper('p_norm', **locals())
                out = helper.create_variable_for_type_inference(
                    dtype=helper.input_dtype()
                )
    
                helper.append_op(
                    type='p_norm',
                    inputs={'X': input},
                    outputs={'Out': out},
                    attrs=attrs,
                )
                return out
    
        def inf_norm(
            input, porder=None, axis=axis, keepdim=False, asvector=False, name=None
        ):
            if in_dygraph_mode():
                out = _C_ops.abs(input)
                if porder == np.float64('inf'):
                    return _C_ops.max(out, axis, keepdim)
                else:
                    return _C_ops.min(out, axis, keepdim)
            else:
                helper = LayerHelper('inf_norm', **locals())
                out = helper.create_variable_for_type_inference(
                    dtype=helper.input_dtype()
                )
                helper.append_op(
                    type='abs', inputs={'X': input}, outputs={'Out': out}
                )
                reduce_out = helper.create_variable_for_type_inference(
                    dtype=helper.input_dtype()
                )
    
                reduce_all = (
                    True if axis is None or axis == [] or asvector else False
                )
                axis = axis if axis is not None and axis != [] else [0]
    
                reduce_type = (
                    'reduce_max' if porder == np.float64('inf') else 'reduce_min'
                )
                helper.append_op(
                    type=reduce_type,
                    inputs={'X': out},
                    outputs={'Out': reduce_out},
                    attrs={
                        'dim': axis,
                        'keep_dim': keepdim,
                        'reduce_all': reduce_all,
                    },
                )
    
                return reduce_out
    
        def p_matrix_norm(input, porder=1.0, axis=axis, keepdim=False, name=None):
            """
            NOTE:
                This function actually treats the matrix as flattened vector to calculate vector norm instead of matrix norm.
            """
            if in_dygraph_mode():
                abs_out = _C_ops.abs(input)
                pow_out = _C_ops.pow(abs_out, porder)
                sum_out = _C_ops.sum(pow_out, axis, None, keepdim)
                out = _C_ops.pow(sum_out, float(1.0 / porder))
                return out
    
            block = LayerHelper('norm', **locals())
            out = block.create_variable_for_type_inference(
                dtype=block.input_dtype()
            )
            abs_out = block.create_variable_for_type_inference(
                dtype=block.input_dtype()
            )
            block.append_op(
                type='abs', inputs={'X': input}, outputs={'Out': abs_out}
            )
            pow_out = block.create_variable_for_type_inference(
                dtype=block.input_dtype()
            )
    
            block.append_op(
                type='pow',
                inputs={'X': abs_out},
                outputs={'Out': pow_out},
                attrs={'factor': porder},
            )
            sum_out = block.create_variable_for_type_inference(
                dtype=block.input_dtype()
            )
            block.append_op(
                type='reduce_sum',
                inputs={'X': pow_out},
                outputs={'Out': sum_out},
                attrs={
                    'dim': axis,
                    'keep_dim': keepdim,
                    'reduce_all': True if axis is None else False,
                },
            )
            block.append_op(
                type='pow',
                inputs={'X': sum_out},
                outputs={'Out': out},
                attrs={'factor': float(1.0 / porder)},
            )
            return out
    
        if axis is None and p is not None:
            if isinstance(p, str):
                if p == "fro":
                    return frobenius_norm(x, dim=axis, keepdim=keepdim, name=name)
                else:
                    raise ValueError(
                        f"only valid string values are 'fro', found {p}"
                    )
            elif isinstance(p, (int, float)):
                return vector_norm(
                    x,
                    porder=p,
                    axis=axis,
                    keepdim=keepdim,
                    asvector=True,
                    name=name,
                )
            else:
                raise ValueError(
                    f"only valid p type is string or float, found {type(p)}"
                )
    
        if isinstance(axis, tuple):
            axis = list(axis)
        if isinstance(axis, list) and len(axis) == 1:
            axis = axis[0]
    
        # calculate vector norm, where axis is int or list with only one integer
        if isinstance(axis, int):
            if isinstance(p, str):
                if p == "fro":
                    return vector_norm(
                        x,
                        porder=2,
                        axis=axis,
                        keepdim=keepdim,
                        asvector=False,
                        name=name,
                    )
    
                else:
                    raise ValueError(
                        f"only valid string values are 'fro', found {p}"
                    )
            elif isinstance(p, (int, float)):
                return vector_norm(
                    x,
                    axis=axis,
                    porder=p,
                    keepdim=keepdim,
                    asvector=False,
                    name=name,
                )
            else:
                raise ValueError(
                    "unspport p for p-order vector norm. except float, found {}".format(
                        p
                    )
                )
E               ValueError: unspport p for p-order vector norm. except float, found None

Could someone please help me and let me know what am I doing wrong? Thank you!

[lululxvi]

@lijialin03 @HydrogenSulfate

[HydrogenSulfate]

I have been experimenting with multiple backends but for some reason, I cannot get the mean_l2_relative_error loss to work with the paddle backend -

import deepxde as dde
import numpy as np
import paddle


# all the lines below error out
dde.losses.mean_l2_relative_error(np.array([[1, 1]]), np.array([[2, 2]]))
dde.losses.mean_l2_relative_error(1, 2)
dde.losses.mean_l2_relative_error(paddle.to_tensor(1.0), paddle.to_tensor(2.0)))

Error -

ValueError                                Traceback (most recent call last)
Cell In[6], line 1
----> 1 dde.losses.mean_l2_relative_error(np.array([[1, 1]]), np.array([[2, 2]]))

File ~/Code/SciML/deepxde/deepxde/losses.py:27, in mean_l2_relative_error(y_true, y_pred)
     26 def mean_l2_relative_error(y_true, y_pred):
---> 27     return bkd.reduce_mean(bkd.norm(y_true - y_pred, axis=1) / bkd.norm(y_true, axis=1))

File ~/Code/SciML/deepxde/deepxde/backend/paddle/tensor.py:176, in norm(x, ord, axis, keepdims)
    175 def norm(x, ord=None, axis=None, keepdims=False):
--> 176     return paddle.linalg.norm(x, p=ord, axis=axis, keepdim=keepdims)

File ~/Library/Python/3.9/lib/python/site-packages/paddle/tensor/linalg.py:593, in norm(x, p, axis, keepdim, name)
    584         return vector_norm(
    585             x,
    586             axis=axis,
   (...)
    590             name=name,
    591         )
    592     else:
--> 593         raise ValueError(
    594             "unspport p for p-order vector norm. except float, found {}".format(
    595                 p
    596             )
    597         )
    598 # calculate matrix norm, where axis is list with two integers
    599 elif isinstance(axis, list) and len(axis) == 2:

ValueError: unspport p for p-order vector norm. except float, found None

Full stack trace -

Full trace
Could someone please help me and let me know what am I doing wrong? Thank you!

First, mean_l2_relative_error is an function in dde module, which means those method should accpet batched Tensor object of their own backend, rather than python builtin float/int or np.ndarray.

Second, there do exist a bug in norm function in paddle backend, and I fix it in #1314 (@lululxvi please help to review and merge). So you can add two line as below

and then this correct code is work.

dde.losses.mean_l2_relative_error(paddle.randn([4, 2]), paddle.randn([4, 2]))

[lululxvi]

#1314 Merged.

[Saransh-cpp]

Thank you for the help and the quick fix, @lululxvi @HydrogenSulfate!