CODEWITHSUNDEEP
rdataframedplyr
psysky
psysky

Reputation: 3195

correlation matrix with p-value in R

Suppose I want conduct correlation matrix

library(dplyr)

data(iris)
iris %>% 
  select_if(is.numeric) %>%
  cor(y =iris$Petal.Width, method = "spearman") %>%  round(2)

now we see

              [,1]
Sepal.Length  0.83
Sepal.Width  -0.29
Petal.Length  0.94
Petal.Width   1.00

i want that statistical significant correlation were marked by * where

*<0,05
**<0,01
*** <0,001

ho to do it?

Upvotes: 4

Views: 3941

Answers (3)

jay.sf
jay.sf

Reputation: 72593

You could adapt corstarsl() to your needs. 

corFun <- function (x) {
  library(Hmisc)
  x <- as.matrix(x)
  R <- rcorr(x, type="spearman")$r
  p <- rcorr(x, type="spearman")$P
  stars <- ifelse(p < 0.001, "***", ifelse(p < 0.01, "** ", 
                                             ifelse(p < 0.05, "* ", " ")))
  R <- format(round(cbind(rep(-1.11, ncol(x)), R), 2))[, -1]
  Rnew <- matrix(paste(R, stars, sep = ""), ncol = ncol(x))
  diag(Rnew) <- paste(diag(R), " ", sep = "")
  rownames(Rnew) <- colnames(x)
  colnames(Rnew) <- paste(colnames(x), "", sep = "")
  Rnew <- as.matrix(Rnew)
  Rnew <- as.data.frame(Rnew)
  return(Rnew)
}

Yielding

> data.frame(r=corFun(iris[, -5])[, 4])
                    r
Sepal.Length  0.83***
Sepal.Width  -0.29***
Petal.Length  0.94***
Petal.Width     1.00

Upvotes: 4

camille
camille

Reputation: 16832

Here are two tidyverse options that both make use of tidy from broom. Using tidy will pull out the estimates and p-values, so you don't have to do that manually. I made a vector of breaks for the different significance levels you want to show, so you can use cut to cut and label the p-values easily; keeping this in a named vector also makes it more repeatable.

The first time I used cor.test, which pipes into the tidy.htest method. The second time I used rcorr from Hmisc, which pipes into the tidy.rcorr method.

In the first case, I gathered the data frame into a long format to compare each measure against Petal.Width; in the second case, which required a matrix, I used the full dataset and then filtered for either column containing Petal.Width.

library(tidyverse)

sig_breaks <- c(zero = 0, "***" = 0.001, "**" = 0.01, "*" = 0.05, NS = Inf)

iris %>%
  as_tibble() %>%
  select_if(is.numeric) %>%
  gather(key = measure, value = value, -Petal.Width) %>%
  group_by(measure) %>%
  do(mtx = cor.test(.$value, .$Petal.Width, method = "spearman")) %>%
  broom::tidy(mtx) %>%
  mutate(stars = cut(p.value, breaks = sig_breaks, include.lowest = T, labels = names(sig_breaks)[2:5]))
#> # A tibble: 3 x 7
#> # Groups:   measure [3]
#>   measure      estimate statistic  p.value method        alternative stars
#>   <chr>           <dbl>     <dbl>    <dbl> <fct>         <fct>       <fct>
#> 1 Petal.Length    0.938    35061. 8.16e-70 Spearman's r… two.sided   ***  
#> 2 Sepal.Length    0.834    93208. 4.19e-40 Spearman's r… two.sided   ***  
#> 3 Sepal.Width    -0.289   725048. 3.34e- 4 Spearman's r… two.sided   ***

iris %>%
  select_if(is.numeric) %>%
  as.matrix() %>%
  Hmisc::rcorr(type = "spearman") %>%
  broom::tidy() %>%
  filter(column1 == "Petal.Width" | column2 == "Petal.Width") %>%
  mutate(stars = cut(p.value, breaks = sig_breaks, include.lowest = T, labels = names(sig_breaks)[2:5]))
#>        column1     column2   estimate   n      p.value stars
#> 1 Sepal.Length Petal.Width  0.8342888 150 0.0000000000   ***
#> 2  Sepal.Width Petal.Width -0.2890317 150 0.0003342981   ***
#> 3 Petal.Length Petal.Width  0.9376668 150 0.0000000000   ***

Created on 2018-05-20 by the reprex package (v0.2.0).

Upvotes: 3

www
www

Reputation: 39154

A solution using . We can convert the data frame to long format, create list column using nest, and then use map to perform cor.test for each subset. After that, map_dbl can extract the P value by specifying the name "p.value". dat1 is the final output.

library(tidyverse)

data(iris)
dat1 <- iris %>% 
  select_if(is.numeric) %>%
  gather(Column, Value, -Petal.Width) %>%
  group_by(Column) %>%
  nest() %>%
  mutate(Cor = map(data, ~cor.test(.x$Value, .x$Petal.Width, method = "spearman"))) %>%
  mutate(Estimate = round(map_dbl(Cor, "estimate"), 2), 
         P_Value = map_dbl(Cor, "p.value"))

dat1
# # A tibble: 3 x 5
#   Column       data               Cor         Estimate  P_Value
#   <chr>        <list>             <list>         <dbl>    <dbl>
# 1 Sepal.Length <tibble [150 x 2]> <S3: htest>    0.83  4.19e-40
# 2 Sepal.Width  <tibble [150 x 2]> <S3: htest>   -0.290 3.34e- 4
# 3 Petal.Length <tibble [150 x 2]> <S3: htest>    0.94  8.16e-70

If you don't need the list columns, you can use select to remove them.

dat1 %>% select(-data, -Cor)
# # A tibble: 3 x 3
#   Column       Estimate  P_Value
#   <chr>           <dbl>    <dbl>
# 1 Sepal.Length    0.83  4.19e-40
# 2 Sepal.Width    -0.290 3.34e- 4
# 3 Petal.Length    0.94  8.16e-70

Now we can use mutate and case_when to add the label to show significance.

dat2 <- dat1 %>%
  select(-data, -Cor) %>%
  mutate(Significance = case_when(
    P_Value < 0.001  ~ "*** <0,001",
    P_Value < 0.01   ~ "** <0,01",
    P_Value < 0.05   ~ "*<0,05",
    TRUE             ~ "Not Significant"
  ))
dat2
# # A tibble: 3 x 4
#   Column       Estimate  P_Value Significance
#   <chr>           <dbl>    <dbl> <chr>       
# 1 Sepal.Length    0.83  4.19e-40 *** <0,001  
# 2 Sepal.Width    -0.290 3.34e- 4 *** <0,001  
# 3 Petal.Length    0.94  8.16e-70 *** <0,001

Upvotes: 6

Related Questions