Unexpected behavior of WeakPDELibrary with STLSQ

[odannis]

Hi,

With the STLSQ optimizer (even with the WeakPDELibrary), I am expecting a coefficient smaller than the threshold 0.1 in ps.STLSQ(threshold=0.1) to be set to 0. Why is it not the case? Is it a bug ?

Thank you,

from pysindy.utils import lorenz
import pysindy as ps
from scipy.integrate import solve_ivp
import numpy as np
# Seed the random number generators for reproducibility
np.random.seed(100)

# Initialize integrator keywords for solve_ivp to replicate the odeint defaults
dt = 0.0005
t_train = np.arange(0, 40, dt)
x0_train = [-8, 8, 27]
t_train_span = (t_train[0], t_train[-1])
x_train = solve_ivp(
    lorenz, t_train_span, x0_train, t_eval=t_train
).y.T

ode_lib = ps.WeakPDELibrary(
    function_library=ps.PolynomialLibrary(degree=2,include_bias=True),
    spatiotemporal_grid=t_train,
    is_uniform=True,
)
optimizer = ps.STLSQ(threshold=0.1)
model = ps.SINDy(feature_library=ode_lib, optimizer=optimizer)
model.fit(x_train)
model.print(precision=10)

I got

(x0)' = -9.9656275630 x0 + 9.9915793731 x1 + -0.0009549384 x0 x2
(x1)' = -0.4914523527 1 + 26.4876595133 x0 + -0.6301465003 x1 + 0.0935151844 x2 + -0.1684472961 x0^2 + 0.1414109986 x0 x1 + -0.9570524373 x0 x2
(x2)' = -2.6578034968 x2 + 0.9958077661 x0 x1

I would expect

(x0)' = -9.9656275630 x0 + 9.9915793731 x1 + **0** x0 x2
(x1)' = -0.4914523527 1 + 26.4876595133 x0 + -0.6301465003 x1 + 0.0935151844 x2 + -0.1684472961 x0^2 + 0.1414109986 x0 x1 + -0.9570524373 x0 x2
(x2)' = -2.6578034968 x2 + 0.9958077661 x0 x1

I am using the last master branch : git checkout 'master@{2024-01-30 18:30:00}'

[Jacob-Stevens-Haas]

How interesting. This is because of unbiasing. STLSQ, like many optimizers, applies a regularizer (a ridge penalty) as well as thresholding. After the optimizer completes, the superclass BaseOptimizer runs one final regression without regularization on the remaining terms.

Before unbiasing, that term is -0.1484984947 x0 x2, which is why it survived thresholding.

To disable this step, set unbias=False in the optimizer.

[fberonvera]

I have been experiencing the same behavior as @odannis. In my case:

optimizer = ps.STLSQ(
threshold=3,
alpha=.05,
normalize_columns=True,
unbias=False)

produces (for some kdv data):

STLSQ model:
(x0)' = -3.763 x0_1 + -0.178 x0_111 + -0.303 x0x0_1 + -0.011 x0^2x0_1 + -0.010 x0x0_111 + -0.002 x0^2x0_111

which is tpretty off but at least I would have expected all coefficients smaller than 3 to be set to 0? Thank you.