CODEWITHSUNDEEP
tensorflowmachine-learningkerasdeep-learning
John App
John App

Reputation: 55

Why is my loss in Keras not changing when training my model?

I have been trying to make a Keras model to find patterns within a numerical dataset of mine. I have changed loss and optimizers many times without any change in the loss. I also have removed/added layers and changed the number of neurons in layers and still get no change in the loss.

The model is:

from keras.models import Sequential
from keras.layers import Dense

import numpy

numpy.random.seed(7)

dataset = numpy.loadtxt("data.csv", delimiter=",")

X = dataset[:, :-1]
Y = dataset[:, -1]

print(X)

# create model
model = Sequential()
model.add(Dense(18, input_dim=18, activation='tanh'))
model.add(Dense(36, activation='relu'))
model.add(Dense(72, activation='relu'))
model.add(Dense(72, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(16, activation='relu'))
model.add(Dense(1, activation='softmax'))

# Compile model
model.compile(loss='mean_squared_error', optimizer='adam')

# Fit the model
model.fit(X, Y, epochs=100, batch_size=35)


# save model
model.save('tried.h5')

I have also changed the epochs and batch size with no effect on the loss.

Here are the logs:

Using TensorFlow backend.

Printing X Data 

[[1.19539070e+01 1.72686310e+01 2.24426384e+01 ... 1.73570000e-04
  4.35710000e-04 9.55710000e-04]
 [1.20239086e+01 1.45762539e+01 2.13278122e+01 ... 1.78570000e-04
  4.06430000e-04 9.17860000e-04]
 [2.30696812e+01 1.82697601e+01 2.13278122e+01 ... 1.15000000e-04
  3.75710000e-04 9.17860000e-04]
 ...
 [2.83583431e+01 2.38079319e+01 2.81154442e+01 ... 1.13570000e-04
  3.20710000e-04 6.65000000e-04]
 [4.34185066e+01 2.17990398e+01 2.81154442e+01 ... 1.12860000e-04
  3.37140000e-04 6.65000000e-04]
 [5.71823807e+01 2.19225960e+01 3.02071724e+01 ... 6.42900000e-05
  3.56430000e-04 6.45000000e-04]]
Epoch 1/100
342420/342420 [==============================] - 15s 45us/step - loss: 0.4945
Epoch 2/100
342420/342420 [==============================] - 15s 44us/step - loss: 0.4945
Epoch 3/100
342420/342420 [==============================] - 15s 43us/step - loss: 0.4945
Epoch 4/100
342420/342420 [==============================] - 15s 44us/step - loss: 0.4945
Epoch 5/100
342420/342420 [==============================] - 15s 44us/step - loss: 0.4945
Epoch 6/100
342420/342420 [==============================] - 15s 44us/step - loss: 0.4945
Epoch 7/100
342420/342420 [==============================] - 14s 42us/step - loss: 0.4945
Epoch 8/100
234500/342420 [===================>..........] - ETA: 4s - loss: 0.4946

The data most certainly does have a slight/decent pattern which the model should be able to pick up on.

Can anyone recommend any changes in order to make the model actually fit the data or spot any errors?

Thank you!

Upvotes: 2

Views: 230

Answers (1)

Peteris
Peteris

Reputation: 3325

A classifier for a single class

The code that you've written ends with a classifier for a single class only, namely the final layer of the model

model.add(Dense(1, activation='softmax'))

has a single neuron, and the softmax function which will ensure that the total of all the neurons in this layer will be equal to 1; commonly used for classifiers to interpret the output as probabilities of the classes.

So, no matter what the weights are there, this network will always output 1.0 for all inputs.

Perhaps you'd want a sigmoid or relu or even linear activation, depending on the numerical distribution of your Y values.

Upvotes: 1

Related Questions