CODEWITHSUNDEEP
r
PJP
PJP

Reputation: 642

reducing the number of comparisons in pairwise.t.test

This is a question stimulated by a previous one I asked: using tapply/dapply etc for t.tests

I have a data frame from an interlab study as follows http://pastebin.com/AD57AYD1

Essentially lab=Laboratory, mat=material, fab=strength, thick=thickness

I want t.test data to compare each lab for each type of material. I.e., for mat=v, I want to run a t.test to compare lab B against lab S. Similarly for materials c, n and l.

My previous question was about using plyr to allow me to run t.tests for each of these combinations. However, it was pointed out that there is the issue of multiple comparisons to consider.

I have tried to use the paired.t.test function on my data, but it did too many comparisons (i.e., it did a t-test of lab B nitrile versus lab S vinyl - which is irrelevant. I called it like this:

pairwise.t.test(interlab$fab,interaction(interlab$mat,interlab$lab),paired=FALSE, pool.sd=FALSE)

and it gave me

> pairwise.t.test(interlab$fab,interaction(interlab$mat,interlab$lab),paired=FALSE, pool.sd=FALSE)

Pairwise comparisons using t tests with non-pooled SD 

    data:  interlab$fab and interaction(interlab$mat, interlab$lab) 

    c.B     l.B     n.B     v.B     c.S     l.S     n.S    
    l.B 0.54484 -       -       -       -       -       -      
    n.B 3.8e-07 1.9e-06 -       -       -       -       -      
    v.B 0.93881 0.22393 3.6e-07 -       -       -       -      
    c.S 0.00576 0.93881 1.2e-05 0.00026 -       -       -      
    l.S 0.00067 0.48601 2.5e-05 4.6e-05 0.89883 -       -      
    n.S 4.3e-12 2.2e-10 0.92366 5.4e-12 6.7e-10 7.7e-10 -      
    v.S 0.93881 0.93881 1.9e-06 0.31885 0.01217 0.00169 1.3e-10

    P value adjustment method: holm

I am concerned that the adjusted p-values in this are wrong, because we were not comparing material n with l, or l with c - we are always lookiung at the same material when tested in both labs (i.e., material 'l' in lab "B" and "S").

Is there any way to subset/group the data so that the appropriate call to pairwise.t.test gives me the following comparisons only ?

        c.B     l.B     n.B     v.B     c.S     l.S     n.S    
    l.B -       -       -       -       -       -       -      
    n.B -       -       -       -       -       -       -      
    v.B -       -       -       -       -       -       -      
    c.S 0.00576 -       -       -       -       -       -      
    l.S -       0.48601 -       -       -       -       -      
    n.S -       -       0.92366 -       -       -       -      
    v.S -       -       -       0.31885 -       -       -

Regards Pete

EDIT: after comments from @John

Whilst it doesn't seem possible to use the pairwise.t.test function in that manner, the previous solution from @droopy can be utilised in a call to the p.adjust function:

 
> FUN<- function(x) {
   t.test(x[,"fab"] ~ x[,"lab"])$p.value
}

res<-ddply(interlab, .(mat), FUN)
res$adjpvalue<-p.adjust(res$V1)

res

mat           V1   adjpvalue
1   c 0.0004798071 0.001919228
2   l 0.0607510365 0.121502073
3   n 0.1847312857 0.184731286
4   v 0.0354274420 0.106282326

Thanks to @John and @droopy for their help in this.

Upvotes: 0

Views: 1310

Answers (1)

John
John

Reputation: 23758

There's no way to do what you ask but what you can do is run your 4 t-tests separately, or extract the p-values that you do want to check from pairwise.t.test when adjustment is set to none, and use the function p.adjust to correct them.

The decision to run pairwise.t.test versus separate tests comes down to using a pooled variance estimate across all of your groups or separate variances. You should decide on that beforehand.

As an aside, did you run an ANOVA? From the looks of your p-values I'm betting that you needn't run any of these tests to simply state from the ANOVA result the pattern of findings.

Upvotes: 2

Related Questions