CODEWITHSUNDEEP
rdataframedplyrtidyverse
Rakhim Rakhimov
Rakhim Rakhimov

Reputation: 109

Confidence interval and p.values for difference between means with summarize function and tidyverse

I am trying to figure out how to turn a data frame from long to wide, while grouping by two variables (diamond cut and colors D and F from diamonds df) and summarizing some key features of the data at the same time.

Specifically, I am trying to get the difference between two means, 95% CI and p-values around that difference.

Here is an example of my desired output table (in red is what I am trying to accomplish).

Sample code below, showing how far I've gotten:

library(tidyverse)

# Build summary data

diamonds <- diamonds %>% 
  select(cut, depth, color) %>% 
  filter(color == "F" | color == "D") %>% 
  group_by(cut, color) %>% 
  summarise(mean = mean(depth), #calculate mean & CIs
            lower_ci = mean(depth) - qt(1- 0.05/2, (n() - 1))*sd(depth)/sqrt(n()),
            upper_ci = mean(depth) + qt(1- 0.05/2, (n() - 1))*sd(depth)/sqrt(n()))

# Turn table from long to wide

diamonds <- dcast(as.data.table(diamonds), cut ~ color, value.var = c("mean", "lower_ci", "upper_ci"))

# Rename & calculate the mean difference

diamonds <- diamonds %>%
  rename(
    Cut = cut,
    Mean.Depth.D = mean_D,
    Mean.Depth.F = mean_F,
    Lower.CI.Depth.D = lower_ci_D,
    Lower.CI.Depth.F = lower_ci_F,
    Upper.CI.Depth.D = upper_ci_D,
    Upper.CI.Depth.F = upper_ci_F) %>% 
  mutate(Mean.Difference = Mean.Depth.D - Mean.Depth.F)

# Re-organize the table

diamonds <- subset(diamonds, select = c(Cut:Mean.Depth.F, Mean.Difference, Lower.CI.Depth.D:Upper.CI.Depth.F))

#Calculate the CIs (upper and lower) and p.values for mean difference for each cut and insert them into the table.

?

I think I am supposed to calculate the CIs and p-values mean difference in depth between colors D and F at some point before I summarize, but not exactly sure how.

Thanks for the input.

Upvotes: 1

Views: 1398

Answers (1)

George Savva
George Savva

Reputation: 5336

To get comparisons of means (with t-tests) for D and F colours across different values for cut, this is what you would need to do:

library(broom)

diamonds %>% 
   filter(color %in% c("D", "F")) %>% 
   group_by(cut) %>% 
   do( tidy(t.test(data=., depth~color)))

Upvotes: 2

Related Questions