CODEWITHSUNDEEP
pythonmachine-learningneural-networkkeras
quil
quil

Reputation: 417

Basic Neural Net Predictions?

I'm looking at this very basic neural net from https://machinelearningmastery.com/tutorial-first-neural-network-python-keras/. I replaced the data used with just a random array of numbers and random labels.

I'm assuming since the input is random, the predictions should be around 0.50, plus or minus a bit. However, when I do this, I get 

[0.49525392, 0.49652839, 0.49729034, 0.49670222, 0.49342978, 0.49490061, 0.49570397, 0.4962129, 0.49774086, 0.49475089, 0.4958384, 0.49506786, 0.49696651, 0.49869373, 0.49537542, 0.49613148, 0.49636957, 0.49723724]

which is around 0.50, but never goes over. It does it for whatever random seed I use, so it isn't just coincidental, either. Any explanations for this behavior? 

# Create first network with Keras
from keras.models import Sequential
from keras.layers import Dense
import numpy as np

np.random.seed(90)

X_train = np.random.rand(18,61250)
X_test = np.random.rand(18,61250)
Y_train = np.array([0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 1.0, 
    0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0,])
Y_test = np.array([1.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 
    1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0,])

_, input_size = X_train.shape

# create model
model = Sequential()
model.add(Dense(12, input_dim=input_size, init='uniform', activation='relu'))
model.add(Dense(8, init='uniform', activation='relu'))
model.add(Dense(1, init='uniform', activation='sigmoid'))

# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# calculate predictions
predictions = model.predict(X_test)
preds = [x[0] for x in predictions]
print(preds)

# Fit the model
model.fit(X_train, Y_train, epochs=100, batch_size=10,  verbose=2, validation_data=(X_test,Y_test))

Upvotes: 0

Views: 123

Answers (1)

lmartens
lmartens

Reputation: 1502

I don't know if this accurately answers your question, but I was playing around with your code and decided to try something. Your X data is generated between 0 and 1, so I tried generating it between 0 and 10. This is a sample of the resulting prediction:

[0.53419214, 0.55088341, 0.53190422, 0.52382213, 0.53469753, 0.53098464,
0.51968938, 0.53249627, 0.52852863, 0.52497149, 0.52816379, 0.5457474,
0.52565753, 0.5276686, 0.52042121, 0.52128422, 0.52535951, 0.52730507]

As you can see, it now produces results over 0.5. Because you predict the output before any training has taken place, the prediction will be done with random weights. Could it be that the network hasn't adjusted yet to the distribution of the input vector?

These are the predictions after training:

[0.43440229, 0.48104468, 0.49194154, 0.4766106, 0.50065982, 0.47388917,
0.51052755, 0.50618082, 0.48478326, 0.4846094, 0.50018799, 0.4800632,
0.4181695, 0.48307362, 0.5063237, 0.50420266, 0.47321039, 0.44235682]

The predictions are now more or less balanced. I get this kind of output with both input distributions. I think it is a matter of the randomly initialized network being very dependent on the distribution of your input data. After training it normalizes.

Upvotes: 2

Related Questions