Why are there discrepancies in partial eta squared values between SPSS and R packages, rstatix and DescTools?

Question

I am running a 2 (Gender) x 2 (Advice) between-subjects ANOVA, and both R and SPSS reported the same ANOVA statistics: for Advice: F = 372.012, effect df = 1, error df = 661; and for Gender x Advice: F = 45.449, effect df = 1, error df = 2.

When calculating the partial eta squared, R (across multiple R packages: rstatix and DescTools) reported 0.221 for Advice and 0.031 for Gender x Advice. But, both SPSS and using Lakens' effect size spreadsheet (https://osf.io/ixgcd/) resulted in 0.360 for Advice and 0.064 for Gender x Advice.

Does R calculate partial eta squared values in a different way than the standard?

Here is a sample data set with just the necessary variables to test the problem: https://docs.google.com/spreadsheets/d/15AIyIfTi9YgMWM5FTl163uddPx1E19xfaU-vMDRlJuI/edit?usp=sharing

Here is the code I used in RStudio:

# load packages
library(haven)
library(rstatix)
library(DescTools)

# read in data
sample_data <- read_sav([insert file location])

# gather Perception1 and Perception2 into 2 groups
sample_data <- sample_data %>% 
gather(key = "Advice", value = "MaleDom", Perception1,  
      Perception2) %>% 
convert_as_factor(ResponseId, Advice)

# rstatix
# compute anova
anova <- aov(MaleDom ~ Gender*Advice, data = sample_data)

# partial eta squared
partial_eta_squared(anova)

# DescTools
# partial eta squared
EtaSq(anova, type = 2, anova = FALSE)

Here is the syntax I used in SPSS:

GLM Perception1 Perception2 BY Gender
/WSFACTOR=advice 2 Polynomial 
/METHOD=SSTYPE(3)
/POSTHOC=Gender(BTUKEY) 
/PLOT=PROFILE(Gender*advice)
/PRINT=DESCRIPTIVE ETASQ HOMOGENEITY 
/CRITERIA=ALPHA(.05)
/WSDESIGN=advice 
/DESIGN=Gender.

Note: I am using SPSS version 26 and R version 3.6.3. I have a Windows 10 with a 64-bit operating system.

Answer 1

Thanks for providing sample data. You get different partial-eta squared values for three possible reasons:

1. In SPSS, you specify that Advice is a repeated measures factor whereas in R you treat it as a between-subjects factor.
2. You are using polynomial contrasts for your within-subjects factor Advice in SPSS whereas in R you are using treatment contrasts.
3. You are using type III sum of squares in SPSS and type I sum of squares in R.

You can specify a repeated measures ANOVA (split-plot design) in R like this:

anova <- aov(MaleDom ~ Gender*Advice + Error(ResponseId/Advice), data=sample_data)

Then

EtaSq(anova, type=1, anova=FALSE)    # note type=1, not 3!

Results in 0.2659604 for partial eta-squared for Advice (sample data). This is equal to the output from SPSS with this syntax:

GLM Perception1 Perception2 BY Gender
  /WSFACTOR=Advice 2 Simple
  /MEASURE=MaleDom
  /CONTRAST(Gender)=Deviation(1)
  /METHOD=SSTYPE(1)
  /PRINT=ETASQ
  /CRITERIA=ALPHA(.05)
  /WSDESIGN=Advice
  /DESIGN=Gender.

Note that partial_eta_squared() from package rstatix does not deal with repeated measures aov() objects, but eta_squared(anova)$Eta_Sq_partial from package effectsize gives the same output as EtaSq() from DescTools.

Answer 2

Use this for defining your data:

 d.dat <- structure(list(ResponseId = c(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 
 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 
 27, 28, 29, 30), Gender = c("Woman", "Woman", "Woman", "Woman", 
 "Woman", "Woman", "Woman", "Woman", "Woman", "Woman", "Woman", 
 "Woman", "Woman", "Woman", "Woman", "Man", "Man", "Man", "Man", 
 "Man", "Man", "Man", "Man", "Man", "Man", "Man", "Man", "Man", 
 "Man", "Man"), Perception1 = c(3.33, 4, 3.67, 1.33, 5.33, 4, 
 6.67, 3.33, 4, 3.67, 4.33, 3.33, 5.33, 1, 2, 6.67, 6.33, 5, 5.33, 
 7, 5, 4.67, 4.33, 6, 5.33, 4, 4.33, 4, 7, 3.33), Perception2 = c(6, 
 6.33, 4, 5, 7, 6, 5, 6.67, 4.67, 5, 5.67, 7, 4, 6, 5.67, 4.67, 
 6.33, 6, 5, 4.67, 5, 6, 4, 5.33, 4, 5, 5.67, 4.67, 6, 6.33)), class = "data.frame",
 row.names = c(NA, -30L))