Sequential model doesn't have outputs

[stsievert]

Shouldn't this code work?

from sklearn.datasets import make_classification
from scikeras.wrappers import KerasClassifier
import tensorflow as tf

def model():
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Dense(8))
    model.add(tf.keras.layers.Dense(1))
    return model

X, y = make_classification(n_features=8)
est = KerasClassifier(model=model, loss="sparse_categorical_crossentropy")
est.fit(X, y=y)

This throws a ValueError: object of type NoneType [self.model_.outputs] has no len().

Full traceback

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
~/Downloads/_junk2.py in <module>
     11 X, y = make_classification(n_features=8)
     12 est = KerasClassifier(model=model, loss="sparse_categorical_crossentropy")
---> 13 est.fit(X, y=y)

~/anaconda3/envs/scikeras/lib/python3.7/site-packages/scikeras/wrappers.py in fit(self, X, y, sample_weight, **kwargs)
   1375             sample_weight = 1 if sample_weight is None else sample_weight
   1376             sample_weight *= compute_sample_weight(class_weight=self.class_weight, y=y)
-> 1377         super().fit(X=X, y=y, sample_weight=sample_weight, **kwargs)
   1378         return self
   1379

~/anaconda3/envs/scikeras/lib/python3.7/site-packages/scikeras/wrappers.py in fit(self, X, y, sample_weight, **kwargs)
    739             epochs=getattr(self, "fit__epochs", self.epochs),
    740             initial_epoch=0,
--> 741             **kwargs,
    742         )
    743

~/anaconda3/envs/scikeras/lib/python3.7/site-packages/scikeras/wrappers.py in _fit(self, X, y, sample_weight, warm_start, epochs, initial_epoch, **kwargs)
    855         X = self.feature_encoder_.transform(X)
    856
--> 857         self._check_model_compatibility(y)
    858
    859         self._fit_keras_model(

~/anaconda3/envs/scikeras/lib/python3.7/site-packages/scikeras/wrappers.py in _check_model_compatibility(self, y)
    541             # we recognize the attribute but do not force it to be
    542             # generated
--> 543             if self.n_outputs_expected_ != len(self.model_.outputs):
    544                 raise ValueError(
    545                     "Detected a Keras model input of size"

TypeError: object of type 'NoneType' has no len()

[adriangb]

Hmm, let me take a look.

[adriangb]

I think you need to add an Input layer:

def model():
    model = tf.keras.Sequential()
    modeo.add(tf.keras.layers.Input(shape=(8,)))
    model.add(tf.keras.layers.Dense(8))  # this does not have to be 8
    ...

Also, related to #206 , it seems like "sparse_categorical_crossentropy" requires the output to have 2 units. "binary_crossentropy" works with 1 output unit.

[adriangb]

Did #207 (comment) solve your problem @stsievert ? Happy to dig deeper otherwise.

[stsievert]

Yes, this code works:

from sklearn.datasets import make_classification
from scikeras.wrappers import KerasClassifier
import tensorflow as tf

def model():
    model = tf.keras.Sequential()
    model.add(tf.keras.layers.Input(shape=(8,)))
    model.add(tf.keras.layers.Dense(8))
    model.add(tf.keras.layers.Dense(1))
    return model

X, y = make_classification(n_features=8)
est = KerasClassifier(model=model, loss="binary_crossentropy")
est.fit(X, y=y)

I think this error should be caught.

[adriangb]

We can leave this open to track it. Ideally, we can come up with a larger category of "Keras Model does not have inputs or outputs".

By the way, I believe that the reason Keras allows this is because Model is just a layer, and thus can be chained, like I did in #200

[stsievert]

This should be easy to catch:

With model.add(tf.keras.layers.Input(shape=(8,))): est.model_.input raises an AttributeError, as does est.model_.output ("'Sequential' object has no attribute _nested_outputs")

Without model.add(tf.keras.layers.Input(shape=(8,))): est.model_.input) is a KerasTensor.

[adriangb]

I agree, it shouldn't be too hard to catch. I'll cook up an implementation once we resolve some of the outstanding PRs, especially #143 . Unless you think this should be incorporated into / superseed #143?

[stsievert]

Eh, I like specific and narrow PRs. Let's keep them separate.

[invexed]

Hi, I'm encountering a similar issue when trying to wrap a TensorFlow Decision Forests model:

from sklearn.datasets import make_classification
import tensorflow_decision_forests as tfdf
from scikeras.wrappers import KerasClassifier

X, y = make_classification(1000, 20, n_informative=10, random_state=0)
model = tfdf.keras.CartModel()
model.compile()
clf = KerasClassifier(model=model)
clf.fit(X, y)

Traceback (most recent call last):
  File "/home/username/dev/tfdf-sklearn/main.py", line 10, in <module>
    clf.fit(X, y)
  File "/home/username/.local/lib/python3.9/site-packages/scikeras/wrappers.py", line 1453, in fit
    super().fit(X=X, y=y, sample_weight=sample_weight, **kwargs)
  File "/home/username/.local/lib/python3.9/site-packages/scikeras/wrappers.py", line 725, in fit
    self._fit(
  File "/home/username/.local/lib/python3.9/site-packages/scikeras/wrappers.py", line 888, in _fit
    self._check_model_compatibility(y)
  File "/home/username/.local/lib/python3.9/site-packages/scikeras/wrappers.py", line 528, in _check_model_compatibility
    if self.n_outputs_expected_ != len(self.model_.outputs):
TypeError: object of type 'NoneType' has no len()

Is there a way to get this to work?
Thanks.

[adriangb]

Does that model have an input layer? Can you post the results of model.summary()?

[invexed]

I trained a tfdf.keras.CartModel using a different dataset and the start of model.summary() showed

Model: "cart_model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
=================================================================
Total params: 1
Trainable params: 0
Non-trainable params: 1
_________________________________________________________________
Type: "RANDOM_FOREST"
Task: CLASSIFICATION
Label: "__LABEL"

[invexed]

So I suppose the answer to your question is no, the model does not have an input layer. Are such models compatible with SciKeras?

[adriangb]

Correct. But Keras allows you to compose models, so you can use your pretrained model as an layer in a new model (kinda) and then just add an input layer before it (and maybe an output layer, depending on your model). Take a look at this SO question: https://datascience.stackexchange.com/a/21620.

Let me know if that works or if it gives you trouble, I can try to write an example closer to your use case if needed.

[invexed]

inputs = Input(shape=(20,))
initial_model = tfdf.keras.CartModel()
outputs = Flatten()(initial_model(inputs))
model = Model(inputs=inputs, outputs=outputs)
model.compile(loss='mse')

The above seems to work, although I get a warning printed:

Warning:  The model was called directly (i.e. using `model(data)` instead of using `model.predict(data)`) before being trained. The model will only return zeros until trained. The output shape might change after training Tensor("inputs:0", shape=(None, 20), dtype=float32)

Have I done something wrong? Do you have to pre-train the initial_model before you can add it as a layer?