leakie
leakie

Reputation: 1

Attribute error with ELMo embedding for CNN model

Having an issue with implementing ELMo embedding with a CNN model. I originally has some tokenization and padding but removed these steps as I am not sure they are needed when using pre-trained embedding (please correct me if I'm wrong). I'm having an attribute error but not sure what it means at all or how to fix it. I tried adding the x_train['tweet'].astype(str) section but that's led to the same error. I know using standard tensorflow.keras instead of tf_keras is an option however I was obtaining different errors before that where the model summary was blank.

The splitting of the data:

#importing data split
from sklearn.model_selection import train_test_split

x = seeker_df[['tweet']]
y = seeker_df['BinaryNumTarget']

#splitting into validation and test - 85% of dataset if train and validation, 15% for test
x_train_val, x_test, y_train_val, y_test = train_test_split(x,
                                                            y, 
                                                            test_size =0.15,
                                                            stratify = y, # sampling used to ensure class distribution
                                                            random_state = 42)

#splitting into train and val - 70% for training, 15% for validation
x_train, x_val, y_train, y_val = train_test_split(x_train_val, 
                                                  y_train_val, 
                                                  test_size =0.1765,
                                                  stratify = y_train_val, 
                                                  random_state=42)
#splitting creates 70% training set, 15% validation set, 15% test set

x_train = x_train['tweet'].astype(str)
x_val = x_val['tweet'].astype(str)
x_test = x_test['tweet'].astype(str)

Packages:

from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
import tensorflow_hub as hub
import tensorflow as tf
#!pip install tf_keras
import tf_keras as tfk

from tf_keras.src.engine.sequential import Sequential
from tf_keras.src.layers.convolutional.conv1d import Conv1D
from tf_keras.src.layers.pooling.global_max_pooling1d import GlobalMaxPooling1D
from tf_keras.src.layers.core.dense import Dense
from tf_keras.src.layers.reshaping.reshape import Reshape
from tf_keras.src.layers.reshaping.flatten import Flatten

The embedding:

#obtaining elmo emmbedding
elmo = hub.KerasLayer("https://tfhub.dev/google/elmo/2", 
                      trainable = False, 
                      name = 'elmo_embedding', 
                      input_shape = [],
                      dtype=(tf.string))

#setting up parameters
max_length = 200
no_of_filters = 250
kernel_size = 3
hidden_dims = 250
batch_size = 32
epochs = 2

The model build:

model = tfk.Sequential()
model.add(elmo)
model.add(Reshape((1024, 1)))

model.add(Conv1D(no_of_filters, kernel_size, activation ='relu'))
model.add(Conv1D(no_of_filters, kernel_size, activation ='relu'))
model.add(GlobalMaxPooling1D())
model.add(Flatten())
model.add(Dense(hidden_dims, activation = 'relu'))

model.add(Dense(1, activation = 'sigmoid'))

model.summary()

model.compile(optimizer='adam',
                  loss='binary_crossentropy',
                  metrics=['accuracy'])


model_history = model.fit(x_train, y_train,
                        validation_data=(x_val, y_val),
                        batch_size= 32, 
                        epochs= 5)

The error:

AttributeError                            Traceback (most recent call last)
Cell In[28], line 23
     15 model.compile(optimizer='adam',
     16                   loss='binary_crossentropy',
     17                   metrics=['accuracy'])
     19 model.summary()
---> 23 model_history = model.fit(x_train, y_train,
     24                         validation_data=(x_val, y_val),
     25                         batch_size= 32, 
     26                         epochs= 5)


AttributeError: in user code:
File "C:---tf_keras\src\engine\training.py", line 1398, in train_function  *
    return step_function(self, iterator)
File "C:---\tf_keras\src\engine\training.py", line 1381, in step_function  **
    outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:---\tf_keras\src\engine\training.py", line 1370, in run_step  **
    outputs = model.train_step(data)
File "C:---\tf_keras\src\engine\training.py", line 1151, in train_step
    self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
File "C:---\tf_keras\src\optimizers\optimizer.py", line 622, in minimize
    grads_and_vars = self.compute_gradients(loss, var_list, tape)
File "C:---\tf_keras\src\optimizers\optimizer.py", line 280, in compute_gradients
    grads = tape.gradient(loss, var_list)

AttributeError: 'NoneType' object has no attribute 'outer_context'

Upvotes: 0

Views: 31

Answers (0)

Related Questions