Diffusion Stable Model: Exception encountered when calling CLIPEncoderLayer.call().

[Muhammadyousafrana]

Issue Type

Documentation Bug

Source

source

Keras Version

3.6.0

Custom Code

Yes

OS Platform and Distribution

No response

Python version

3.12

GPU model and memory

T4 Tesla

Current Behavior?

TypeError: Exception encountered when calling CLIPEncoderLayer.call().

Standalone code to reproduce the issue or tutorial link

!pip install -q --upgrade keras-cv
!pip install -q --upgrade keras  # Upgrade to Keras 3.
import time
import keras_cv
import tensorflow as tf
from tensorflow import keras
import matplotlib.pyplot as plt
model = keras_cv.models.StableDiffusion(img_height=512,
                                        img_width=512,
                                        jit_compile=False)
images = model.text_to_image("photograph of an astronaut riding a horse", batch_size=3)

Relevant log output

/usr/local/lib/python3.10/dist-packages/keras/src/layers/layer.py:1381: UserWarning: Layer 'clip_encoder_layer_1' looks like it has unbuilt state, but Keras is not able to trace the layer `call()` in order to build it automatically. Possible causes:
1. The `call()` method of your layer may be crashing. Try to `__call__()` the layer eagerly on some test input first to see if it works. E.g. `x = np.random.random((3, 4)); y = layer(x)`
2. If the `call()` method is correct, then you may need to implement the `def build(self, input_shape)` method on your layer. It should create all variables used by the layer (e.g. by calling `layer.build()` on all its children layers).
Exception encountered: ''Exception encountered when calling CLIPAttention.call().

pred must not be a Python bool

Arguments received by CLIPAttention.call():
  • inputs=tf.Tensor(shape=(None, 77, 768), dtype=float32)
  • attention_mask=None''
  warnings.warn(
/usr/local/lib/python3.10/dist-packages/keras/src/layers/layer.py:391: UserWarning: `build()` was called on layer 'clip_encoder_layer_1', however the layer does not have a `build()` method implemented and it looks like it has unbuilt state. This will cause the layer to be marked as built, despite not being actually built, which may cause failures down the line. Make sure to implement a proper `build()` method.
  warnings.warn(
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-f098215f8cd6> in <cell line: 1>()
----> 1 images = model.text_to_image("photograph of an astronaut riding a horse", batch_size=3)
      2 
      3 def plot_images(images):
      4   plt.figure(figsize=(20,20))
      5   for i in range(len(images)):

6 frames
/usr/local/lib/python3.10/dist-packages/keras_cv/src/models/stable_diffusion/text_encoder.py in call(self, inputs, attention_mask)
    134         if attention_mask is None and self.causal:
    135             length = ops.shape(inputs)[1]
--> 136             attention_mask = ops.triu(
    137                 ops.ones((1, 1, length, length), dtype=self.compute_dtype)
    138                 * -float("inf"),

TypeError: Exception encountered when calling CLIPEncoderLayer.call().

Could not automatically infer the output shape / dtype of 'clip_encoder_layer_1' (of type CLIPEncoderLayer). Either the `CLIPEncoderLayer.call()` method is incorrect, or you need to implement the `CLIPEncoderLayer.compute_output_spec() / compute_output_shape()` method. Error encountered:

Exception encountered when calling CLIPAttention.call().

pred must not be a Python bool

Arguments received by CLIPAttention.call():
  • inputs=tf.Tensor(shape=(None, 77, 768), dtype=float32)
  • attention_mask=None

Arguments received by CLIPEncoderLayer.call():
  • args=('<KerasTensor shape=(None, 77, 768), dtype=float32, sparse=False, name=keras_tensor_3>',)
  • kwargs=<class 'inspect._empty'>