CODEWITHSUNDEEP
rmetricsrocauc
screechOwl
screechOwl

Reputation: 28189

R / Metrics / AUC : How can these numbers give an AUC of 1?

I have the following toy dataset:

df1<-structure(list(X1 = c(1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 1, 0, 
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1), X2 = c(0.564666517055852, 
0.993248174442609, 0.517237113309667, 0.0128217854167603, 0.952654357126895, 
0.958073009436008, 0.860038905846366, 0.425314512801637, 0.809327038447625, 
0.985049417726494, 0.165982081954436, 0.517237113309667, 0.00211852090504078, 
0.296898500479658, 0.994690775408805, 0.999991149759367, 0.999949243479285, 
0.999979994962211, 0.409697759931823, 0.999995828877373, 0.999991594894354, 
0.999999834424374, 0.952641245900919, 0.999998774453881, 0.999999777896636, 
0.999998864433372, 0.998786297471059, 0.999927421881167, 0.998265361329274, 
0.999550929839182, 0.999900216754163, 0.999912135543067, 0.999999924775596, 
0.996227950775217, 0.998265981873947, 0.999959584436354, 0.999993039255167, 
0.99968139946193, 0.999999997308486, 0.999999458017638, 0.999996417856357, 
0.99958403590535, 0.999998891765696, 0.999999624757926, 0.999818190766803, 
0.999997979863151, 0.999974432439759, 0.996227950775217, 0.999999771762929, 
0.983441425608786, 0.99999843468322)), .Names = c("X1", "X2"), row.names = c(NA, 
-51L), class = "data.frame")

when I use the auc() function from the Metrics package it tells me that the score is 1. 

> Metrics::auc(df1$X1, df1$X2)
[1] 1

This doesn't seem correct. Any suggestions?

Upvotes: 2

Views: 196

Answers (2)

AGS
AGS

Reputation: 14498

By plotting the ROC of your data, you'll see that the AUC is 1.0

enter image description here

Upvotes: 0

David
David

Reputation: 9405

That is correct. You only have 3 zeros, which is certainly going to bring on issues, but just look here (all zeros have the lower predicted probabillities than ones):

> dat[order(dat[,2]),]
   X1          X2
13  0 0.002118521
4   0 0.012821785
11  0 0.165982082
14  1 0.296898500
19  1 0.409697760
8   1 0.425314513
3   1 0.517237113
12  1 0.517237113
1   1 0.564666517
9   1 0.809327038
7   1 0.860038906
23  1 0.952641246
5   1 0.952654357
6   1 0.958073009
50  1 0.983441426
10  1 0.985049418
2   1 0.993248174
15  1 0.994690775
34  1 0.996227951
48  1 0.996227951
29  1 0.998265361
35  1 0.998265982
27  1 0.998786297
30  1 0.999550930
42  1 0.999584036
38  1 0.999681399
45  1 0.999818191
31  1 0.999900217
32  1 0.999912136
28  1 0.999927422
17  1 0.999949243
36  1 0.999959584
47  1 0.999974432
18  1 0.999979995
16  1 0.999991150
21  1 0.999991595
37  1 0.999993039
20  1 0.999995829
41  1 0.999996418
46  1 0.999997980
51  1 0.999998435
24  1 0.999998774
26  1 0.999998864
43  1 0.999998892
40  1 0.999999458
44  1 0.999999625
49  1 0.999999772
25  1 0.999999778
22  1 0.999999834
33  1 0.999999925
39  1 0.999999997

Upvotes: 4

Related Questions