ValueError: A KerasTensor is symbolic: it's a placeholder for a shape an a dtype. It doesn't have any actual numerical value. You cannot convert it to a NumPy array.

[accioharshita]

Hey, so I've downloaded the preprocessing & encoder layer of BERT in order to build a simple email classification model. When I'm finally building my model to pass the training data it throws this error. Can someone tell me what's wrong?

[LeviBen]

I have the same problem. With the URL was working fine, but with the model working locally, for some reason it crashes

[SDSCodes22]

@accioharshita Hi, I was having the same problem. In fact, I was using the exact same code as you. I managed to solve my problem by importing Bert through the keras_nlp library. Here is the code I ended up with:

text_input = tf.keras.layers.Input(shape=(), dtype=tf.string)
preprocessor = keras_nlp.models.BertPreprocessor.from_preset("bert_base_en_uncased",trainable=True)
encoder_inputs = preprocessor(text_input)
encoder = keras_nlp.models.BertBackbone.from_preset("bert_base_en_uncased")
outputs = encoder(encoder_inputs)
pooled_output = outputs["pooled_output"]      # [batch_size, 768].
sequence_output = outputs["sequence_output"]  # [batch_size, seq_length, 768].

[BismaAyaz]

@SoumyaCodes2020 can you please let me know how you saved model with this approach. I'm using this approach model3.save("model3.keras")
model3 = keras.models.load_model("model3.keras") but getting error

No vocabulary has been set for WordPieceTokenizer. Make sure to pass a `vocabulary` argument when creating the layer.

[leinonk1]

My problem disappeared when I installed tensorflow 2.15.1 and tensorflow-text 2.15.0, instead of the newest 2.16.0. tensorflow-hub version was 0.16.1. (my comment below does not apply to this solution)

@SoumyaCodes2020 how many trainable params do you have in the model if you use that approach (when you call model.summary())? It seems that this approach leads to the Bert layer parameters being trainable (even if trainable=False). I had to solve it by looping through all layers in the encoder and setting the layers as non-trainable:
for layer in encoder.layers: layer.trainable = False

[arc2226]

@accioharshita Hi, I was having the same problem. In fact, I was using the exact same code as you. I managed to solve my problem by importing Bert through the keras_nlp library. Here is the code I ended up with:

text_input = tf.keras.layers.Input(shape=(), dtype=tf.string)
preprocessor = keras_nlp.models.BertPreprocessor.from_preset("bert_base_en_uncased",trainable=True)
encoder_inputs = preprocessor(text_input)
encoder = keras_nlp.models.BertBackbone.from_preset("bert_base_en_uncased")
outputs = encoder(encoder_inputs)
pooled_output = outputs["pooled_output"]      # [batch_size, 768].
sequence_output = outputs["sequence_output"]  # [batch_size, seq_length, 768].

This worked for me too.

[AlbertoMQ]

This is not an appropriate solution to the problem.

[dnydlk]

I found solution from this issue from tensorflow_hub repo issues

import os
os.environ['TF_USE_LEGACY_KERAS']='1'

import tensorflow as tf
import tensorflow_hub as hub
import tensorflow_text as text

I put the first block of code before I run all the imports and it worked

[dsmabulage]

My problem disappeared when I installed tensorflow 2.15.1 and tensorflow-text 2.15.0, instead of the newest 2.16.0. tensorflow-hub version was 0.16.1. (my comment below does not apply to this solution)

@SoumyaCodes2020 how many trainable params do you have in the model if you use that approach (when you call model.summary())? It seems that this approach leads to the Bert layer parameters being trainable (even if trainable=False). I had to solve it by looping through all layers in the encoder and setting the layers as non-trainable: for layer in encoder.layers: layer.trainable = False

Thank you, you saved my time