EarlyStoppingCallback Exception

[MichaelCarrik]

in tsai 0.4.0. I use EarlyStoppingCallback for LSTM_FCNPlus. but it raise
File "/home/research/code/FTD_Research5/Model/TSAI/RNNResModel.py", line 285, in
learn.fit(500, learnRate,cbs=[EarlyStoppingCallback(monitor='mse_loss', min_delta=0.1, patience=2)] )
File "/usr/local/lib/python3.11/site-packages/fastai/learner.py", line 272, in fit
self._with_events(self._do_fit, 'fit', CancelFitException, self._end_cleanup)
File "/usr/local/lib/python3.11/site-packages/fastai/learner.py", line 207, in with_events
try: self(f'before{event_type}'); f()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastai/learner.py", line 180, in call
def call(self, event_name): L(event_name).map(self._call_one)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastcore/foundation.py", line 163, in map
def map(self, f, *args, **kwargs): return self._new(map_ex(self, f, *args, gen=False, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastcore/basics.py", line 934, in map_ex
return list(res)
^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastcore/basics.py", line 919, in call
return self.func(*fargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastai/learner.py", line 184, in _call_one
for cb in self.cbs.sorted('order'): cb(event_name)
^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/fastai/callback/core.py", line 64, in call
except Exception as e: raise modify_exception(e, f'Exception occured in {self.__class__.__name__} when calling event {event_name}:\n\t{e.args[0]}', replace=True)

[vrodriguezf]

This error typically occurs because the EarlyStoppingCallback in fastai/tsai expects the metric you specify in monitor (in this case, 'mse_loss') to be among the learner's metrics or have an attribute by that name.Is your learner a TSRegressor with MSE loss?

[MichaelCarrik]

This error typically occurs because the EarlyStoppingCallback in fastai/tsai expects the metric you specify in monitor (in this case, 'mse_loss') to be among the learner's metrics or have an attribute by that name.Is your learner a TSRegressor with MSE loss?

yes my code is "learn.fit(500, learnRate,cbs=[EarlyStoppingCallback(monitor='mse_loss', min_delta=0.1, patience=2)] )"

[SSHurtado]

This seems to be a configuration issue on your side, rather than an LSTM_FCNPlus bug.

EarlyStoppingCallback(monitor='mse_loss') only works if mse_loss is actually being logged. By default, even when MSE is used as the loss function, fastai usually logs it as valid_loss, not mse_loss.

You could try monitoring valid_loss instead:

learn.fit(
    500,
    learnRate,
    cbs=[EarlyStoppingCallback(monitor='valid_loss', min_delta=0.1, patience=2)]
)

Or explicitly add MSE as a metric to monitor.