CODEWITHSUNDEEP
pythonkerasscikit-learngoogle-colaboratory
An old man in the sea.
An old man in the sea.

Reputation: 1508

Why is Google Colab taking so long for learning these simple neural networks?

I'm trying to run the followin code on google colab:

import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from keras import models
from keras import layers
from keras.datasets import boston_housing
from sklearn.model_selection import KFold
from sklearn.model_selection import ShuffleSplit
from sklearn.model_selection import GridSearchCV
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.metrics import mean_squared_error

sns.set()

(train_data,train_targets),(test_data,test_targets)=boston_housing.load_data()

mean=np.mean(train_data)
std=np.std(train_data)

train_data_norm=(train_data-mean)/std
test_data_norm=(test_data-mean)/std

def build_model():
    model=models.Sequential()
    model.add(layers.Dense(64,activation="relu",
                          input_shape=(train_data_norm.shape[1],)))
    model.add(layers.Dense(64,activation="relu"))
    model.add(layers.Dense(1))
    model.compile(optimizer='rmsprop',loss="mse",metrics=["mae"])
    return model 


model=KerasRegressor(build_fn=build_model,epochs=30,verbose=0)
param_grid = {"epochs":range(1,11)}

kf1=KFold(n_splits=5, random_state=None, shuffle=False) #n_splits = number of folds
kf2=KFold(n_splits=5, random_state=1, shuffle=True)
ss = ShuffleSplit(n_splits=5,test_size=0.20,random_state=1)

grid_model_KFFalse=GridSearchCV(model,param_grid,cv=kf1,n_jobs=-1,scoring='neg_mean_squared_error')
grid_model_KFTrue=GridSearchCV(model,param_grid,cv=kf2,n_jobs=-1,scoring='neg_mean_squared_error')
grid_model_SS=GridSearchCV(model,param_grid,cv=ss,n_jobs=-1,scoring='neg_mean_squared_error')

listKFFalse=[]
listKFTrue=[]
listSS=[]

for i in range(1,21):
  grid_model_KFFalse.fit(train_data, train_targets)
  grid_model_KFTrue.fit(train_data, train_targets)
  grid_model_SS.fit(train_data, train_targets)
  mseKFFalse=mean_squared_error(grid_model_KFFalse.predict(test_data),test_targets)
  mseKFTrue=mean_squared_error(grid_model_KFTrue.predict(test_data),test_targets)
  mseSS=mean_squared_error(grid_model_SS.predict(test_data),test_targets)
  listKFFalse=np.append(listKFFalse,[mseKFFalse])
  listKFTrue=np.append(listKFTrue,[mseKFTrue])
  listSS=np.append(listSS,[mseSS])

You can check it here.

I've done on my laptop 1 run on a jupyter notebook, instead of the 21, and the part where it's slowest is in the 'for' block, but that's to be expected.

The code compiles without error, since I've already managed to run it for 5 cycles/simulations without problems.

However, it's taking a very long time... I wonder why it is. Isn't google colab supposed to use GPU(or TPU) when learning neural networks with keras? I've already changed the Runtime type to GPU or TPU, and it's still very slow.

Upvotes: 0

Views: 520

Answers (2)

Alex Metsai
Alex Metsai

Reputation: 1950

From the FAQ

Seems too good to be true. What are the limitations?

Colab resources are not guaranteed and not unlimited, and the usage limits sometimes fluctuate. This is necessary for Colab to be able to provide resources for free. For more details, see Resource Limits

Upvotes: 2

Allen Wang
Allen Wang

Reputation: 301

Regarding TPU usage, if you refer to TPUs in Colab you'll see some code snippets for connecting to a TPU:

tpu = tf.distribute.cluster_resolver.TPUClusterResolver() 
tf.config.experimental_connect_to_cluster(tpu)
tf.tpu.experimental.initialize_tpu_system(tpu)
tpu_strategy = tf.distribute.experimental.TPUStrategy(tpu)

In the snippet/colab provided, the TPU isn't being used which is why you see no difference in performance. You would need to migrate from base keras to tf.keras and use distribution strategies if you want to use TPUs.

Upvotes: 0

Related Questions