Comparing Physics-Informed Neural Network architectures (vanilla MLP, Fourier-feature MLP, Modified MLP, self-adaptive weighting) on steady 2D incompressible Navier-Stokes forward problems, evaluated against Ghia et al. (1982) reference solutions for lid-driven cavity flow.
This project benchmarks how different PINN architectures solve the steady 2D incompressible Navier-Stokes equations on the lid-driven cavity problem at Re=100 — the standard PINN-CFD benchmark. Poiseuille channel flow (analytical solution available) is used as a correctness sanity check for the shared NS residual implementation.
The goal is to isolate which architectural choices (input embedding, gated connections, adaptive loss weighting) actually move accuracy and training cost on low-Re laminar flow, where Ghia et al. (1982) provides a trusted tabulated reference for quantitative comparison.
| Architecture | Mechanism | Hypothesis |
|---|---|---|
| Vanilla MLP | Fully-connected tanh MLP, fixed-weight composite loss | Baseline; expected to underresolve cavity recirculation due to spectral bias |
| Fourier MLP (σ=5.0) | Random Fourier embedding, frequency scale σ=5.0 | Over-injects high frequencies; expected worse than vanilla on smooth-ish flow |
| Fourier MLP (σ=1.0) | Random Fourier embedding, frequency scale σ=1.0 | Correct frequency scale should reduce spectral bias and resolve recirculation |
| Modified MLP | Gated encoder branches (Wang et al. 2022) | Fixes gradient pathology; should improve over vanilla without spectral bias risk |
| SA-PINN (vanilla backbone) | Trainable per-point λ weights, minimax training | Tests whether adaptive collocation weighting helps independently of embedding |
| Architecture | u_centerline error | v_centerline error | Final loss | Training time | Converged? |
|---|---|---|---|---|---|
| Vanilla MLP | 0.182 | 0.831 | 1.15e-02 | ~126s | Partial (plateaued) |
| Fourier MLP σ=5.0 | 0.189 | 0.927 | 1.24e-02 | ~163s | Partial |
| Fourier MLP σ=1.0 | 0.037 | 0.177 | 6.0e-03 | ~221s | Yes |
| Modified MLP | 0.052 | 0.275 | 9.65e-03 | ~271s | Yes |
| SA-PINN (vanilla) | >vanilla | >vanilla | — | ~243s | No (λ uniform) |
All errors are relative L2 against Ghia et al. (1982) centerline tabulated data.
Centerline comparison plots: see results/figures/.
pinn_cfd_bench/ # core package: geometry, physics residual, networks, training, eval configs/ # per-problem YAML configs (domain, Re, BC, hyperparameters) reference_data/ # Ghia et al. (1982) tabulated centerline data scripts/ # entrypoints: train.py for single runs tests/ # residual correctness tests (Poiseuille analytical unit test) notebooks/ # final results analysis and plots results/ # logs, checkpoints, figures (generated, not committed)
pip install -r requirements.txt
# Run unit tests first — confirms PDE residual is correct before any training
pytest tests/
# Vanilla MLP on Poiseuille (baseline sanity check)
python scripts/train.py --config configs/poiseuille.yaml
# Vanilla MLP on cavity Re=100 (baseline)
python scripts/train.py --config configs/cavity_re100.yaml
# Fourier MLP on cavity Re=100 (best result so far)
python scripts/train.py --config configs/fourier_mlp_cavity_re100.yaml
# Fourier MLP on Poiseuille (shows embedding hurts smooth flows)
python scripts/train.py --config configs/fourier_mlp_poiseuille.yaml-
Vanilla MLP fails to resolve cavity recirculation. The model correctly anchors boundary conditions but collapses to a low-frequency approximation in the interior, producing 18% u-error and 83% v-error. The centerline v-velocity plot shows a single smooth hump entirely missing the double-lobed recirculation structure Ghia et al. report. This is a textbook manifestation of spectral bias in plain tanh-MLPs.
-
Fourier features fix spectral bias, but σ is critical. With σ=1.0 (matching the solution's characteristic frequency scale), Fourier MLP achieves 5x improvement over vanilla (u: 0.182→0.037, v: 0.831→0.177) and correctly resolves the double-lobed v-velocity structure. With σ=5.0 (too high), performance is worse than vanilla (u: 0.189, v: 0.927) — wrong σ actively harms training. σ is not a minor tuning detail; it determines whether the architecture helps or hurts.It helps to learn better either the low frequency data and high frequency data depending upon the sigma value taken .
-
Modified MLP (Wang et al. 2022) gives robust improvement without hyperparameter sensitivity. The gated encoder architecture(creating two encoders U and V taking spatial and temporal coordinates ) achieves 3.5x improvement over vanilla (u: 0.052, v: 0.275) without requiring frequency-scale tuning. Loss descends steadily throughout training with no plateau, confirming the gating mechanism successfully addresses gradient pathology. Comes second to Fourier σ=1.0 but is more robust — a practitioner who doesn't know the right σ in advance would be better served by Modified MLP than by Fourier features with an untuned σ.
-
SA-PINN adaptive weighting failed on this problem. Despite implementing the minimax training scheme with fixed collocation points and warmup phases, λ weights remained near-uniform (std/mean < 0.003) throughout training. Attributing this to insufficient spatial residual heterogeneity in smooth low-Re flow — the PDE residual is too uniformly small across the domain for the ascent step to learn meaningful point-wise importance weights. SA-PINN's benefit likely requires either higher-Re flows with sharper boundary layers, or much longer training runs than attempted here.
-
Architecture interacts with problem smoothness. On Poiseuille flow (parabolic profile, smooth analytical solution), vanilla MLP achieves 1.6% u-error while Fourier MLP σ=1.0 gives 47% u-error — the embedding over-parameterizes a solution the plain MLP already handles well. Modified MLP's gating does not carry this risk since it adapts connectivity rather than injecting fixed frequency content.
- Only steady, low-Re laminar flow tested. Fourier features and SA-PINN may show larger advantages at higher Re or unsteady flows where residual heterogeneity increases and solution complexity demands higher-frequency representation.
- σ sweep limited to {1.0, 5.0}; a finer sweep may identify a more optimal value and narrow the gap between Fourier MLP and Modified MLP.
- Cavity results evaluated on centerline profiles only (17-point Ghia et al. table); full-field L2 error against a dense FEM reference not computed.
- SA-PINN not tested beyond 5000 Adam steps; longer training with a slower λ learning rate schedule may eventually produce spatial differentiation.
- Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.
- Ghia, U., Ghia, K. N., & Shin, C. T. (1982). High-Re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method.
- Tancik, M., et al. (2020). Fourier features let networks learn high frequency functions in low dimensional domains.
- Wang, S., Teng, Y., & Perdikaris, P. (2021). Understanding and mitigating gradient pathologies in physics-informed neural networks.