why the predict_proba function of sklearn.svm.svc is giving probability greater than 1?

Question

I've got a sklearn.svm.svc (RBF kernel) model trained on two classes containing 140 samples each. The probability is set to true when I tried to predict and the prediction probability of those two classes are varying.

1. For some test samples, it is giving one probability greater than 1
   and other less than one

   e.g ('sample-1': 1.55478334, 'sample-2': 0.999984).

2. In some cases, it is giving both probabilities less than one

   e.g ('sample-1': 0.4182294947776875, 'sample-2': 0.58177035052223113).

Is my model working well or there is some fault in my training or testing. Probability greater then 1Probability less then 1

my code is as follows:

#Training code      
        tcdf512_d1=np.empty(280,(18)),dtype=float)
            lables=np.empty((0))
            model512_d1=SVC(probability=True)
            for img,img2 in map(None,catA,catB):
                if img!=None:
                    tcdf512_d1[k]=img(18 features i.e. skewness,variance, standard deviation etc)
                    k+=1
                    lables=np.append(lables,'Cat-VI')
                    pass
                if img2!=None:
                    tcdf512_d1[k]=img2(18 features i.e. skewness,variance, standard deviation etc)
                    k+=1
                    lables=np.append(lables,'Cat-VII')
                    pass
                if k%50==0:
                    print (k)

            print ("LBP Calculated")
            print (time.strftime('%d/%m/%Y %H:%M:%S'))
            model512_d1.fit(tcdf512_d1,lables)
            tcdf512_d1=None
            lables=None
            k=None
            print ("Model Trained")
            print (time.strftime('%d/%m/%Y %H:%M:%S'))
            joblib.dump(model512_d1,"Cat/momentsCat_6-7_128_d1.pkl",compress=3)
            print ("Model Saved")
            print (time.strftime('%d/%m/%Y %H:%M:%S'))
            model512_d1=None
#Testing Code

    size=128
    Cat_I_II       =  joblib.load("Cat/momentsCat_6-7_128_d1.pkl")
    name1="VII"
    print (name1)
    images_address="Catagory/Testbg/"+name1+"/"
    name1="Cat-"+str(name1)
    test_images = cvutils.imlist(images_address)

    count =images_address.rfind("/")+1
    results1=[]
    print (len(test_images))
    print ("Start Time ")
    print (time.strftime('%d/%m/%Y %H:%M:%S'))
    j=float(len(test_images))
    k=0
#    testdata=[]
    for img3 in test_images:
        results1.append("Image : "+str(img3[count:]))
        results1.append("\n")
    varientarray=[]        
        array=[]
        array.append(img3(18 features i.e. skewness,variance, standard deviation etc))
        print array
        prediction = Cat_I_II.predict(array)[0]
        prob=Cat_I_II.predict_proba(array)[0]
        prob_per_class_dictionary = dict(zip(Cat_I_II.classes_, prob))
        print(prediction,prob_per_class_dictionary)
        results1.append("Result of Cat_I_II is : "+str(prediction) +"\t"+str(prob_per_class_dictionary))
        varientarray.append(prediction)

        print (k)
        print ("Final Result of image "+str(i[count:]) + " is : "+str(collections.Counter(varientarray).most_common(1)[0][0]))
        results1.append("Final Result of image "+str(i[count:]) + " is : "+str(collections.Counter(varientarray).most_common(1)[0][0]))

        if str(i[count:i.index('0')])==collections.Counter(varientarray).most_common(1)[0][0]:
            j-=1
        gc.collect()
        k+=1
    k=float(j*100/len(test_images))
    Accuracy=float((len(test_images)-j)*100/len(test_images))
    print (j)
    print (k)
    print (Accuracy)
    with open("CatResults/_Finalresults.txt", 'a') as f:
        f.write(str("The accuracy for "+str(name1)+" is :"+str(Accuracy)) +"\n")
    results1.append("Incorrect Results are :"+str(j))
    results1.append("The percentage of incorrect result is :"+str(k))
    results1.append("The accuracy is :"+str(Accuracy))
    with open("CatResults/Cat-"+str(name1)+"resultsp2.txt", 'w') as f:
        for s in results1:
            f.write(str(s) +"\n")
    print ("End Time")
    print(time.strftime('%d/%m/%Y %H:%M:%S'))

My results snippets are as follows

Answer 1

Please notice the e-06 or e-08 in those probabilities. Thats equivalent to 10^(-08) in scientific notation. So the above 1 probability you are thinking is very very less.

For example:

2.798594e-06 = 0.000002798594

Similarly,

7.7173288137e-08 = 0.000000077173288137

So when you sum those values you will get 1. If not 1, then it will be something like 0.99999999... . Thats expected due to rounding off of the results displayed.

So the predict_proba results are not inconsistent. They are actually correct.

Now as for why the predicted result does not match with the highest predicted probability, thats described in the documentation and is expected behaviour due to internals of the algorithm. Please look at the documentation:-

• http://scikit-learn.org/dev/modules/svm.html#scores-and-probabilities

The probability estimates may be inconsistent with the scores, in the sense that the “argmax” of the scores may not be the argmax of the probabilities. (E.g., in binary classification, a sample may be labeled by predict as belonging to a class that has probability <½ according to predict_proba.)