model performance when using limited number of data for inverse PINN modeling

[Elyasin97]

Hello, Dr. Lu!

I am currently working on predicting flow velocity in a geometry obtained by subtracting a circle from a polygon. I am using the deepxde library and the PINN method for this task. When I use 1006 data points from my dataset, my predictions closely match the values predicted by CFD. However, I want to use a smaller subset of data, for example, I used 380 data points from my dataset, and unfortunately, the predictions are not acceptable.

My question is: how can I enhance the code to achieve better performance when working with a limited number of data points?
I would greatly appreciate your guidance.
@lululxvi

Here is my code:

import deepxde as dde
import numpy as np
import matplotlib.pyplot as plt
from scipy.io import loadmat
import re
import pandas as pd

#true values
C1true = 1.0
C2true = 0.01

#Load training data
def load_training_data(num):
data = pd.read_excel("data.xlsx",sheet_name='Sheet3')
x = data.iloc[:, 0].values
y = data['y'].values
u = data['U'].values
v = data['V'].values
p = data['Pressure'].values

#Training domain: X × Y = [1, 8] × [−2, 2]
data_domain = np.column_stack((x, y, u, v, p))
idx = np.random.choice(data_domain.shape[0], num, replace=False)
x_train = data_domain[idx, 0:1]
y_train = data_domain[idx, 1:2]
u_train = data_domain[idx, 2:3]
v_train = data_domain[idx, 3:4]
p_train = data_domain[idx, 4:5]
return [x_train, y_train, u_train, v_train, p_train]

#Define Navier Stokes Equations (Static PDEs)
def Navier_Stokes_Equation(x, y):
u = y[:, 0:1]
v = y[:, 1:2]
p = y[:, 2:3]
du_x = dde.grad.jacobian(y, x, i=0, j=0)
du_y = dde.grad.jacobian(y, x, i=0, j=1)
dv_x = dde.grad.jacobian(y, x, i=1, j=0)
dv_y = dde.grad.jacobian(y, x, i=1, j=1)
dp_x = dde.grad.jacobian(y, x, i=2, j=0)
dp_y = dde.grad.jacobian(y, x, i=2, j=1)
du_xx = dde.grad.hessian(y, x, component=0, i=0, j=0)
du_yy = dde.grad.hessian(y, x, component=0, i=1, j=1)
dv_xx = dde.grad.hessian(y, x, component=1, i=0, j=0)
dv_yy = dde.grad.hessian(y, x, component=1, i=1, j=1)
continuity = du_x + dv_y
x_momentum = C1true * (u * du_x + v * du_y) + dp_x - C2true * (du_xx + du_yy)
y_momentum = C1true * (u * dv_x + v * dv_y) + dp_y - C2true * (dv_xx + dv_yy)
return [continuity, x_momentum, y_momentum]

#Define Spatial Domain
geom1 = dde.geometry.geometry_2d.Polygon([[0.0, 2.0], [12.0, 2.0], [12.0, 0.0], [2.0, 0.0], [2.0, 1.0],
[0.0, 1.0]])
geom2 = dde.geometry.geometry_2d.Disk([4,1],0.3)
space_domain = geom1 - geom2

#Get the training data: num
[ob_x, ob_y, ob_u, ob_v, ob_p] = load_training_data(num=380)
ob_xy = np.hstack((ob_x, ob_y))
observe_u = dde.icbc.PointSetBC(ob_xy, ob_u, component=0)
observe_v = dde.icbc.PointSetBC(ob_xy, ob_v, component=1)

#Training datasets and Loss
data = dde.data.TimePDE(
space_domain,
Navier_Stokes_Equation,
[observe_u, observe_v],
num_domain=100,
num_boundary=100,
anchors=ob_xy,
)

#Neural Network setup
layer_size = [2] + [40] * 7 + [3]
activation = "tanh"
initializer = "Glorot uniform"
net = dde.nn.FNN(layer_size, activation, initializer)
model = dde.Model(data, net)

early_stopping = dde.callbacks.EarlyStopping(min_delta=1e-6, patience=5000)

#Compile, train and save model
model.compile("adam", lr=1e-3)
loss_history, train_state = model.train(iterations=60000, callbacks=[early_stopping], display_every=1000,disregard_previous_best=True)
dde.saveplot(loss_history, train_state, issave=True, isplot=True)
model.compile("adam", lr=1e-4)
loss_history, train_state = model.train(iterations=60000, callbacks=[early_stopping], display_every=1000,disregard_previous_best=True)
dde.saveplot(loss_history, train_state, issave=True, isplot=True)

f = model.predict(ob_xy, operator=Navier_Stokes_Equation)
print("Mean residual:", np.mean(np.absolute(f)))

X = space_domain.random_points(1000000)
output = model.predict(X)

u_pred = output[:, 0]
v_pred = output[:, 1]
velocity_pred = np.sqrt(u_pred2 + v_pred2)
p_pred = output[:, 2]
fig = plt.figure(figsize=(30,5))
plt.scatter(X[:, 0], X[:, 1], c=velocity_pred, cmap='jet', s=2)
plt.colorbar()

This is the volcity magnitude profile from CFD

This is my prediction with 1006 number of data

and this is what i achieve when I use 380 number of data

[Elyasin97]

I found the mistakes of the code and it's solved but now i want to save and restore it for next times. I read FAQ and I tried both methods to restore saved model however I got same error as Issue: Get, save, load weights and biases from a trained PINN #103
I would be appriciated if you give your kindly suggests.
@lululxvi

[lululxvi]

I don't understand your error.

[david-e-gomes]

Hello,
I am having a very similar problem. Could you show me how you fixed it?
I would be very thankful because I am stuck.

@Elyasin97