CODEWITHSUNDEEP
rfloating-point-precision
Stephen Turner
Stephen Turner

Reputation: 2604

How to get precise low p-value in R (from F test)

I'm computing a p-value from an F-test in R, and it seems to have trouble displaying anything lower than 1e-16. E.g., for F values from 80 to 90 with degrees of freedom 1,200: 

> 1-pf(80:90,1,200)
 [1] 2.220446e-16 2.220446e-16 1.110223e-16 1.110223e-16 0.000000e+00 0.000000e+00 0.000000e+00
 [8] 0.000000e+00 0.000000e+00 0.000000e+00 0.000000e+00

How can I increase the precision of the pf() function's calculations?

Upvotes: 3

Views: 3726

Answers (2)

Brian Diggs
Brian Diggs

Reputation: 58835

In addition to Jack's practical advice about p-values, the functions are well defined (if not practical) so I'll give the finite-precision mathematical reason this doesn't work.

.Machine$double.eps is 2.220446e-16 and that is the smallest number that you can add to 1 and get something different. So differencing from 1, that is the smallest value you get. 

> pf(80:90,1,200)
 [1] 1 1 1 1 1 1 1 1 1 1 1
> sprintf("%.17f",pf(80:90,1,200))
 [1] "0.99999999999999978" "0.99999999999999978" "0.99999999999999989"
 [4] "0.99999999999999989" "1.00000000000000000" "1.00000000000000000"
 [7] "1.00000000000000000" "1.00000000000000000" "1.00000000000000000"
[10] "1.00000000000000000" "1.00000000000000000"
> sprintf("%a", pf(80:90,1,200))
 [1] "0x1.ffffffffffffep-1" "0x1.ffffffffffffep-1" "0x1.fffffffffffffp-1"
 [4] "0x1.fffffffffffffp-1" "0x1p+0"               "0x1p+0"              
 [7] "0x1p+0"               "0x1p+0"               "0x1p+0"              
[10] "0x1p+0"               "0x1p+0"

However you can use the approximation $1-p = -\ln(p)$ and the fact that you can get the log of the p-values more precisely

> -pf(80:90,1,200,log.p=TRUE)
 [1] 2.540347e-16 1.770938e-16 1.236211e-16 8.640846e-17 6.047690e-17
 [6] 4.238264e-17 2.974043e-17 2.089598e-17 1.470045e-17 1.035491e-17
[11] 7.303070e-18

Upvotes: 4

Jack Aidley
Jack Aidley

Reputation: 20107

p-values this low are meaningless anyway. Firstly, most calculations use slight approximations so the imprecision comes to dominate the result as you tend towards a zero p-value and secondly, and probably more importantly, any tiny deviation of your population from the modelled distribution will overwhelm the accuracy you desire.

Simply quote the p-values as 'p < 0.0001' and be done with it.

Upvotes: 12

Related Questions