Reputation: 1
Having an issue with implementing ELMo embedding with a CNN model. I originally has some tokenization and padding but removed these steps as I am not sure they are needed when using pre-trained embedding (please correct me if I'm wrong). I'm having an attribute error but not sure what it means at all or how to fix it. I tried adding the x_train['tweet'].astype(str) section but that's led to the same error. I know using standard tensorflow.keras instead of tf_keras is an option however I was obtaining different errors before that where the model summary was blank.
The splitting of the data:
#importing data split
from sklearn.model_selection import train_test_split
x = seeker_df[['tweet']]
y = seeker_df['BinaryNumTarget']
#splitting into validation and test - 85% of dataset if train and validation, 15% for test
x_train_val, x_test, y_train_val, y_test = train_test_split(x,
y,
test_size =0.15,
stratify = y, # sampling used to ensure class distribution
random_state = 42)
#splitting into train and val - 70% for training, 15% for validation
x_train, x_val, y_train, y_val = train_test_split(x_train_val,
y_train_val,
test_size =0.1765,
stratify = y_train_val,
random_state=42)
#splitting creates 70% training set, 15% validation set, 15% test set
x_train = x_train['tweet'].astype(str)
x_val = x_val['tweet'].astype(str)
x_test = x_test['tweet'].astype(str)
Packages:
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
import tensorflow_hub as hub
import tensorflow as tf
#!pip install tf_keras
import tf_keras as tfk
from tf_keras.src.engine.sequential import Sequential
from tf_keras.src.layers.convolutional.conv1d import Conv1D
from tf_keras.src.layers.pooling.global_max_pooling1d import GlobalMaxPooling1D
from tf_keras.src.layers.core.dense import Dense
from tf_keras.src.layers.reshaping.reshape import Reshape
from tf_keras.src.layers.reshaping.flatten import Flatten
The embedding:
#obtaining elmo emmbedding
elmo = hub.KerasLayer("https://tfhub.dev/google/elmo/2",
trainable = False,
name = 'elmo_embedding',
input_shape = [],
dtype=(tf.string))
#setting up parameters
max_length = 200
no_of_filters = 250
kernel_size = 3
hidden_dims = 250
batch_size = 32
epochs = 2
The model build:
model = tfk.Sequential()
model.add(elmo)
model.add(Reshape((1024, 1)))
model.add(Conv1D(no_of_filters, kernel_size, activation ='relu'))
model.add(Conv1D(no_of_filters, kernel_size, activation ='relu'))
model.add(GlobalMaxPooling1D())
model.add(Flatten())
model.add(Dense(hidden_dims, activation = 'relu'))
model.add(Dense(1, activation = 'sigmoid'))
model.summary()
model.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
model_history = model.fit(x_train, y_train,
validation_data=(x_val, y_val),
batch_size= 32,
epochs= 5)
The error:
AttributeError Traceback (most recent call last)
Cell In[28], line 23
15 model.compile(optimizer='adam',
16 loss='binary_crossentropy',
17 metrics=['accuracy'])
19 model.summary()
---> 23 model_history = model.fit(x_train, y_train,
24 validation_data=(x_val, y_val),
25 batch_size= 32,
26 epochs= 5)
AttributeError: in user code:
File "C:---tf_keras\src\engine\training.py", line 1398, in train_function *
return step_function(self, iterator)
File "C:---\tf_keras\src\engine\training.py", line 1381, in step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
File "C:---\tf_keras\src\engine\training.py", line 1370, in run_step **
outputs = model.train_step(data)
File "C:---\tf_keras\src\engine\training.py", line 1151, in train_step
self.optimizer.minimize(loss, self.trainable_variables, tape=tape)
File "C:---\tf_keras\src\optimizers\optimizer.py", line 622, in minimize
grads_and_vars = self.compute_gradients(loss, var_list, tape)
File "C:---\tf_keras\src\optimizers\optimizer.py", line 280, in compute_gradients
grads = tape.gradient(loss, var_list)
AttributeError: 'NoneType' object has no attribute 'outer_context'
Upvotes: 0
Views: 31