CODEWITHSUNDEEP
pythontensorflowkerasdeep-learning
Avi
Avi

Reputation: 2170

Deep Learning - Keep getting low accuracy

I'm new to Python (although not to programming - I'm usually programming in JavaScript) and I'm very interested in AI development.
Recently I've been trying to develop a deep learning algorithm by following this article.

My goal is to predict a set of 7 numbers, based on a CSV file that contains a large list, with each row having 7 numbers as well. The order of the list matters.

I ended up having the following code:

from keras.models import Sequential
from keras.layers import Dense
from sklearn.model_selection import train_test_split
from numpy import loadtxt, random

random.seed(seed)

dataset = loadtxt("data/Lotto.csv", delimiter=",", skiprows=1)

X = dataset[:, 0:7]

Y = dataset[:, 6]

(X_train, X_test, Y_train, Y_test) = train_test_split(X, Y, test_size=0.33, random_state=4)

model = Sequential()
model.add(Dense(8, input_dim=7, kernel_initializer="uniform", activation="relu"))
model.add(Dense(6, kernel_initializer="uniform", activation="relu"))
model.add(Dense(1, kernel_initializer="uniform", activation="sigmoid"))

model.compile(loss="binary_crossentropy", optimizer="adam", metrics=["accuracy"])

model.fit(X_train, Y_train, validation_data=(X_test, Y_test), epochs=100, batch_size=5, shuffle=False)

scores = model.evaluate(X_test, Y_test)

print("Accuracy: %.2f%%" %(scores[1] * 100))

After running it in Google Colaboratory, while I'm not getting any errors - I noticed that for each epoch, the loss result doesn't change, and as a result, I keep getting low accuracy (~6%).

What am I doing wrong?

Upvotes: 0

Views: 172

Answers (2)

simon
simon

Reputation: 826

Correct me if I'm wrong, but by the looks of it your input is a list of 7 numbers, and you want to output the 7th number in that list. Now by using a sigmoid activation in your last layer, you're restricting your model output to the interval (0,1). Are you sure that your data is in this interval?

Also, your model is way to complicated for that task. You really need only one dense layer without an activation or bias to be able to do this.

Upvotes: 0

Abhay Kashyap
Abhay Kashyap

Reputation: 1

Try changing optimizer to RMSprop with learning rate of around 0.0001. RMSprop is usually better than most optimizers and gives better accuracy and lesser loss than others. You could alternatively try SGD, which is also a good optimizer. Also increase number of parameters, as more trainable parameters leads to the model being trained with more precision and gives a much accurate prediction. You could update the code to tensorflow 2.x and change the code to :

from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
from sklearn.model_selection import train_test_split
from numpy import loadtxt, random

#Rest of the code
....... 
.......
.......

model = Sequential()
model.add(Dense(64, input_shape=(7,), activation='relu', kernel_initializer='uniform'))
model.add(Dense(64, kernel_initializer="uniform", activation="relu"))
model.add(Dense(1, kernel_initializer="uniform", activation="sigmoid"))

model.compile(loss="binary_crossentropy", optimizer=tf.keras.optimizers.RMSprop(learning_rate=0.0001), metrics=["accuracy"])

model.fit(X_train, Y_train, validation_data=(X_test, Y_test), epochs=100, batch_size=5, shuffle=False)
scores = model.evaluate(X_test, Y_test)
print("Accuracy: %.2f%%" %(scores[1] * 100))

Upvotes: 0

Related Questions