CODEWITHSUNDEEP
pythonneural-networkkeras
user1157751
user1157751

Reputation: 2457

Keras - predict_proba does not add up to 1?

I'm new to Keras, and this is my nn:

model = Sequential()
model.add(Dense(12, activation="relu", input_dim=12, kernel_initializer="normal"))
model.add(Dense(3, activation="sigmoid", kernel_initializer="normal"))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

I have 3 outputs: high, medium, or low.

When I tried to use the predict function, I get the correct classes:

array(['medium', 'low', 'medium', ..., 'low', 'medium', 'low'], dtype=object)

However, when i tried predict_proba function, I would get probabilities that doesn't add up to 1?

array([[  4.93983855e-04,   2.28181913e-01,   9.70774710e-01],
       [  3.14530940e-03,   9.60477769e-01,   3.79752717e-04],
       [  1.40661141e-03,   5.70683666e-02,   9.96348858e-01],
       ..., 
       [  1.29012510e-01,   7.08254218e-01,   2.44960701e-03],
       [  1.69786233e-02,   4.71719205e-02,   9.90665674e-01],
       [  1.28657368e-04,   9.89430904e-01,   9.04915680e-04]], dtype=float32)

The first row is 4.93983855e-04 + 2.28181913e-01 + 9.70774710e-01, the sum is 1.199450606855, which is greater than 1.

I thought the 1st row means: probability of 1st column (4.93983855e-04) is medium?

Upvotes: 0

Views: 3120

Answers (1)

Nassim Ben
Nassim Ben

Reputation: 11543

The sigmoid activation is outputing values between 0 and 1 independently from one another.

If you want probabilities outputs that sums up to 1, use the softmax activation on your last layer, it will normalize the output to sum up to 1.

I hope this helps :)

Upvotes: 8

Related Questions