Léo Henry
Léo Henry

Reputation: 137

Which formula to choose for a mixed model

Edit: I have posted A more detailed question of the following on Cross Validated as I have been told that the following was more appropriate to be posted there.

I'm trying to write a mixed model on repeated data, but I am having a hard time writing the formula.

My database is composed of results of schools for different exams. Each line contains a result from 0 to 100. For each result I know the school, the year it was taken, and the subject of the exam. I separated the schools in two kinds (A and B); my objective is to measure if the kind (A or B) of a school has an influence on its result. However, as I said, I have repeated measures for each school because I have 20 grades by school, one for each year (2014 to 2019) and one for each of four subjects. Here is an example of my data:

       Result School_ID Year    Subject School_type
1      19         1 2015       math           A
2      35         1 2015    english           A
3       4         1 2015    history           A
4      16         1 2015 philosophy           A
5      55         1 2016       math           A
6      62         1 2016    english           A
7      74         1 2016    history           A
8      66         1 2016 philosophy           A
9      32         1 2017       math           A
10     16         1 2017    english           A
11     42         1 2017    history           A
12     52         1 2017 philosophy           A
13     95         1 2018       math           A
14      8         1 2018    english           A
15     35         1 2018    history           A
16     41         1 2018 philosophy           A
17     12         1 2019       math           A
18     40         1 2019    english           A
19     56         1 2019    history           A
20     65         2 2019 philosophy           B
21     12         2 2015       math           B
22     23         2 2015    english           B
23     45         2 2015    history           B
24     90         2 2015 philosophy           B
25      3         2 2016       math           B
26     66         2 2016    english           B
27     51         2 2016    history           B
28     26         2 2016 philosophy           B
29      4         2 2017       math           B

Because I have repeated data, I know I have to use a mixed model using lmer (for instance). However, I am struggling to find a good formula.

Firstly I tried this:

Result ~ School_type * Subject + (1|Year)

But I'm not really sure that is a good formula. I also tried this

Result ~ School_type * Subject + (Year|School_ID)

But it does not give any significant result. At first I thought my variables School_type had no significant influence on Result. Which was reeally unexpected. Then I thought, that by considering the school as a random variable, this formula was suppressing the effect of the school on my results, which obviously would conduct to reduced the effect of School_type because it was attached to the schools.

In other word, I want to understand the effect of characteristics of schools on their results, but if I "suppress" the effect of the individuals, here the schools, in the random effects, I obviously will not get any results.

However, I have the feeling that my first formula:

Result ~ School_type * Subject + (1|Year)

which gives me the expected results, does not take into account the fact that the data are repeated for the schools.

As requested, here is the dput(my_data_frame).

structure(list(result = c(69.9, 72.4, 67.1, 84.4, 84.9, 68, 78.1, 
65.1, 69.9, 77.5, 80.2, 84.7, 89.3, 82.6, 63.8, 40.8, 72.2, 71.4, 
77.2, 79.9, 93.5, 65, 67.7, 91.8, 79.6, 73.4, 80.9, 85.7, 91.8, 
67.1, 66.5, 84.6, 80.8, 87.3, 94, 87.8, 86.2, 36.6, 37.6, 18, 
30, 34.8, 32.5, 21.9, 29, 22.7, 47.3, 70, 84.6, 60.8, 42.1, 18.6, 
49.2, 33.9, 34.9, 47.1, 29.2, 34.5, 70.3, 56, 67.8, 60.9, 50.3, 
40.4, 20.8, 45.4, 57.7, 65, 26.5, 40.1, 36.6, 49.1, 52.4, 22.8, 
46.5, 42.4, 54.1, 51.3, 27.2, 42.9, 47.1, 61.6, 45.1, 86.5, 69.2, 
58.4, 58.4, 46.8, 77, 46.9, 73.1, 50.1, 61.4, 49.2, 75, 75.4, 
53.2, 71.9, 49.5, 27.4, 48.3, 51.9, 68.8, 69, 44.6, 39, 48.3, 
77.5, 59.3, 70.9, 80.1, 73.5, 77.9, 57.3, 76.8, 67, 63.2, 89.8, 
79.5, 70.8, 78.5, 79.4, 79.4, 80.5, 72, 68.6, 91.7, 75.6, 77.2, 
77.8, 73, 85.3, 68.8, 64.6, 88.2, 76.9, 84.7, 88.5, 76.6, 81.8, 
26.6, 30.9, 27.9, 33.5, 27.8, 8.7, 27.8, 31.8, 68, 23.5, 80.2, 
54.8, 42.7, 46.1, 49.6, 27, 35.2, 30, 32, 62.9, 56.6, 70.2, 44.3, 
39.3, 37, 24, 52.2, 44.2, 59.3, 30.4, 33.9, 33.8, 65.7, 42.8, 
36.5, 43.1, 49.7, 34.3, 10.4, 47.4, 43.4, 60.8, 58.5, 93.2, 81.7, 
79.4, 76.8, 66.3, 76.2, 68.2, 45, 46.7, 57.4, 89.6, 81.6, 73.8, 
61.1, 43.2, 41.9, 74.8, 56.9, 71.5, 30.9, 59.6, 39.8, 76.3, 67.3, 
54.5, 78, 92.6, 80.9, 81.7, 64, 75.2, 62.3, 75.9, 87.1, 73.1, 
64.8, 79.3, 85.4, 81.3, 74.4, 90.6, 68.7, 71.3, 92.4, 79.5, 72.1, 
86, 85.7, 90.8, 66.6, 62.8, 84.4, 77.9, 85.6, 86, 80.6, 49.4, 
42.1, 28, 35.2, 32.5, 45.6, 35.6, 38.1, 38.8, 48.3, 32.2, 84.6, 
61.4, 46.7, 68.7, 65, 26.7, 49.2, 55.2, 26.8, 21.8, 55.4, 43.4, 
56, 44, 65.2, 42.4, 41.9, 27.6, 44, 51.1, 63, 32.9, 52.3, 44.8, 
58.1, 44, 15.2, 44.2, 68.8, 53.3, 25, 57.8, 51.7, 60.8, 59.4, 
88.2, 83.6, 74.2, 81.7, 64.8, 63.4, 61.1, 46.8, 47.2, 69.4, 76.3, 
87.9, 78.7, 70.7, 60.8, 34.6, 42.3, 54.7, 62.1, 33.6, 51, 31.1, 
72.5, 66.9, 69.4, 79.5, 80.6, 80.8, 70.5, 72.5, 63.6, 96.6, 78.4, 
90.8, 75.9, 57.1, 76.2, 74.8, 59.9, 61.7, 82.6, 65.9, 73.8, 90.1, 
83.2, 78, 83.5, 83.8, 87, 67.2, 60.7, 79.9, 76.1, 82, 82.2, 84.6, 
48.1, 33, 13.5, 35.7, 42.6, 23.6, 35.4, 40.4, 41.6, 46.5, 24.6, 
78.8, 73.6, 41.1, 68.5, 45.1, 38.6, 45, 78.4, 23.9, 35.6, 55.6, 
53.3, 57.1, 63.2, 67.9, 55.2, 23.5, 41.3, 56, 50.6, 50.6, 44.2, 
44.9, 46.4, 54.6, 41.5, 30, 53.6, 81, 37.2, 48, 56.8, 77.4, 59.2, 
91.8, 77.3, 78.1, 80.8, 63.4, 54.6, 51.3, 63.5, 53.2, 76.3, 72.7, 
79.6, 57.8, 40.2, 66, 58.3, 80.4, 72.8, 53.6, 54.2, 54, 65.3, 
68.4, 79, 75.6, 81.8, 54.4, 68.5, 68.5, 77.9, 84.8, 87.1, 74.9, 
64.9, 74.1, 63.8, 76, 86, 67.9, 81.8, 91.2, 81.2, 72.2, 74.9, 
85.4, 90.8, 71, 64.5, 84.3, 83.3, 84.3, 85.1, 38.5, 46, 44.1, 
49.3, 37.9, 26.9, 36.9, 32.3, 45.2, 51.9, 84.8, 65.9, 53.3, 43.3, 
42.1, 71.4, 42.6, 29.4, 38.6, 49.5, 40.5, 63.8, 46.3, 28.7, 31.7, 
57.1, 66.1, 21.8, 37.6, 49.5, 49.9, 55.7, 53.2, 53.2, 53.2, 40.2, 
57.3, 64.4, 88.3, 74.2, 68.6, 53.4, 70.5, 52.4, 60.6, 49.6, 57.2, 
62.6, 65.5, 66.7, 52.7, 46.7, 51.1, 53.4, 56.3, 73.7, 34.7, 65.1, 
50, 54.8, 48.2, 59.8, 79, 50.1, 53, 54.8, 53.7, 52.8, 72.8, 55.4, 
90, 64.1, 50.3, 2.5, 49.3, 46, 57.9, 42.2, 93.2, 53.1, 51.5, 
80.7, 49.7, 70.9, 63.6, 63.2, 61.8, 59.3, 47.4, 54.9, 77.4, 69.7, 
66.1, 48.8, 72.1, 59.5, 58.7, 54.3, 64.4, 83.9, 52.1, 53.1, 55.1, 
63.3, 61.4, 79.5, 57.5, 68.3, 47.7, 70.4, 61, 53, 55.1, 89.2, 
60.3, 50.2, 83.1, 76.9, 69.7, 64.7, 58.8, 62, 48.5, 46, 66.1, 
74.1, 56, 69, 64.3, 54, 33.9, 58, 78.9, 53.4, 59.7, 59.1, 63.4, 
52.2, 68.5, 56, 96.5, 53.3, 47.9, 60, 66.9, 56.6, 39.7, 93.5, 
47.4, 40.1, 78.1, 63.1, 62.6, 64.3, 57.5, 56.8, 51.9, 48.8, 44.7, 
56.4, 65.2, 70, 56.2, 63.2, 53.4, 44.6, 42, 56.2, 50.3, 50.2, 
53.3, 47.5, 46.7, 64.1, 49.5, 71.9, 57.8, 37.2, 47.9, 46.9, 42.3, 
38.4, 42.9, 46.4, 73.6, 60.6, 53.2, 61, 55.4, 41.7, 47.9, 49.6, 
58.5, 63, 42.3, 65.7, 48.4, 53.5, 53.2, 51.5, 53.9, 47.8, 52.3, 
58.5, 59.9, 59.9, 74, 57.8, 44.1, 61.5, 47, 59.4, 48.9, 40.5, 
74.6, 56.6, 52.9, 72.2, 59.8, 50.1, 46.5, 68.7, 64.1, 64.6, 53.1, 
64.8), school_ID = structure(c(1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 
9L, 10L, 11L, 12L, 13L, 14L, 15L, 16L, 17L, 18L, 19L, 20L, 21L, 
22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 31L, 32L, 33L, 34L, 
35L, 36L, 37L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 
13L, 14L, 15L, 16L, 17L, 18L, 19L, 20L, 22L, 23L, 24L, 25L, 26L, 
27L, 28L, 29L, 31L, 32L, 33L, 34L, 35L, 36L, 37L, 1L, 2L, 3L, 
4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 17L, 18L, 
19L, 20L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 31L, 
32L, 33L, 34L, 35L, 36L, 37L, 1L, 2L, 3L, 4L, 6L, 7L, 8L, 9L, 
10L, 12L, 13L, 14L, 15L, 17L, 18L, 19L, 20L, 22L, 23L, 24L, 25L, 
26L, 27L, 28L, 29L, 30L, 31L, 32L, 33L, 34L, 35L, 36L, 37L, 1L, 
2L, 3L, 4L, 6L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 17L, 18L, 
19L, 20L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 31L, 32L, 33L, 
34L, 35L, 36L, 37L, 1L, 2L, 3L, 4L, 6L, 7L, 8L, 9L, 10L, 11L, 
12L, 13L, 14L, 15L, 17L, 18L, 19L, 20L, 22L, 23L, 24L, 26L, 27L, 
28L, 29L, 30L, 31L, 32L, 33L, 34L, 35L, 36L, 37L, 1L, 2L, 3L, 
4L, 5L, 6L, 7L, 8L, 9L, 10L, 12L, 13L, 14L, 15L, 17L, 18L, 19L, 
20L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 31L, 32L, 
33L, 34L, 36L, 37L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 9L, 10L, 11L, 
12L, 13L, 14L, 15L, 17L, 18L, 19L, 20L, 21L, 22L, 23L, 24L, 25L, 
26L, 27L, 28L, 29L, 30L, 31L, 32L, 33L, 34L, 35L, 36L, 37L, 1L, 
2L, 3L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 17L, 
18L, 19L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 32L, 
33L, 34L, 35L, 36L, 37L, 1L, 2L, 3L, 4L, 6L, 7L, 8L, 9L, 10L, 
11L, 12L, 13L, 14L, 15L, 17L, 18L, 19L, 20L, 21L, 22L, 23L, 24L, 
25L, 26L, 27L, 28L, 29L, 30L, 31L, 32L, 33L, 34L, 36L, 37L, 1L, 
2L, 3L, 4L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 17L, 
18L, 19L, 20L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 
31L, 32L, 33L, 34L, 35L, 36L, 37L, 1L, 2L, 3L, 4L, 6L, 7L, 9L, 
10L, 11L, 12L, 13L, 14L, 15L, 17L, 18L, 19L, 22L, 23L, 24L, 26L, 
27L, 28L, 29L, 30L, 31L, 32L, 33L, 34L, 35L, 36L, 37L, 1L, 2L, 
3L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 17L, 18L, 19L, 
21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 31L, 32L, 33L, 
34L, 36L, 1L, 2L, 3L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 
15L, 17L, 18L, 19L, 22L, 23L, 25L, 26L, 27L, 28L, 29L, 30L, 31L, 
32L, 34L, 35L, 36L, 1L, 2L, 3L, 6L, 7L, 9L, 10L, 11L, 12L, 13L, 
14L, 17L, 18L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 
31L, 32L, 33L, 34L, 35L, 36L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 
9L, 10L, 11L, 12L, 13L, 14L, 15L, 16L, 17L, 18L, 19L, 20L, 21L, 
22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 31L, 32L, 33L, 34L, 
35L, 36L, 37L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 
12L, 14L, 15L, 17L, 18L, 19L, 20L, 21L, 22L, 23L, 24L, 26L, 27L, 
28L, 29L, 30L, 31L, 33L, 34L, 35L, 36L, 37L, 1L, 2L, 3L, 4L, 
5L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 17L, 18L, 19L, 
20L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L, 31L, 32L, 
33L, 34L, 35L, 36L, 37L, 1L, 2L, 3L, 4L, 6L, 7L, 8L, 9L, 10L, 
11L, 12L, 13L, 14L, 15L, 17L, 18L, 19L, 20L, 22L, 23L, 24L, 26L, 
27L, 28L, 29L, 30L, 31L, 33L, 34L, 35L, 36L, 37L, 1L, 2L, 3L, 
6L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 17L, 18L, 19L, 
22L, 23L, 24L, 26L, 27L, 28L, 29L, 30L, 31L, 33L, 34L, 35L, 36L, 
37L), .Label = c("1", "2", "3", "4", "5", "6", "7", "8", "9", 
"10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "20", 
"21", "22", "23", "24", "25", "26", "27", "28", "29", "30", "31", 
"32", "33", "34", "35", "36", "37"), class = "factor"), year = structure(c(5L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 5L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 
5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 
5L, 5L, 5L, 5L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = c("2015", "2016", "2017", 
"2018", "2019"), class = "factor"), subject = structure(c(1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 
2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 
3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 
4L, 4L, 4L, 4L, 4L, 4L, 4L), .Label = c("Math", "History", "Philosiphy", 
"English"), class = "factor"), school_type = structure(c(1L, 
1L, 1L, 3L, 3L, 2L, 2L, 1L, 2L, 2L, 3L, 2L, 3L, 3L, 1L, 1L, 2L, 
2L, 3L, 2L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 1L, 2L, 3L, 
1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 3L, 2L, 2L, 1L, 2L, 2L, 3L, 3L, 
3L, 1L, 1L, 2L, 2L, 3L, 2L, 1L, 2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 
2L, 3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 3L, 2L, 2L, 1L, 2L, 2L, 
3L, 2L, 3L, 3L, 1L, 2L, 2L, 3L, 2L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 
1L, 3L, 1L, 1L, 2L, 3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 2L, 2L, 
1L, 2L, 2L, 2L, 3L, 3L, 1L, 2L, 2L, 3L, 2L, 1L, 2L, 3L, 1L, 1L, 
2L, 1L, 3L, 1L, 1L, 2L, 3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 2L, 
1L, 2L, 2L, 3L, 2L, 3L, 3L, 1L, 2L, 2L, 3L, 2L, 1L, 2L, 3L, 1L, 
1L, 2L, 1L, 3L, 1L, 2L, 3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 2L, 
2L, 1L, 2L, 2L, 3L, 2L, 3L, 3L, 1L, 2L, 2L, 3L, 2L, 1L, 2L, 3L, 
1L, 2L, 1L, 3L, 1L, 1L, 2L, 3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 
3L, 2L, 2L, 1L, 2L, 2L, 2L, 3L, 3L, 1L, 2L, 2L, 3L, 2L, 1L, 1L, 
2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 1L, 2L, 3L, 1L, 3L, 3L, 1L, 1L, 
1L, 3L, 3L, 2L, 2L, 2L, 2L, 3L, 2L, 3L, 3L, 1L, 2L, 2L, 3L, 2L, 
1L, 1L, 2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 1L, 2L, 3L, 1L, 3L, 3L, 
3L, 1L, 1L, 1L, 3L, 2L, 2L, 1L, 2L, 2L, 3L, 2L, 3L, 3L, 1L, 2L, 
2L, 3L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 2L, 3L, 1L, 3L, 
3L, 3L, 1L, 1L, 1L, 3L, 2L, 2L, 1L, 2L, 2L, 3L, 2L, 3L, 3L, 1L, 
2L, 2L, 3L, 2L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 1L, 2L, 
3L, 1L, 3L, 3L, 1L, 1L, 1L, 3L, 2L, 2L, 1L, 2L, 2L, 3L, 2L, 3L, 
3L, 1L, 2L, 2L, 3L, 2L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 
1L, 2L, 3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 2L, 2L, 2L, 2L, 3L, 
2L, 3L, 3L, 1L, 2L, 2L, 3L, 1L, 2L, 3L, 1L, 2L, 1L, 3L, 1L, 1L, 
2L, 3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 2L, 2L, 1L, 2L, 2L, 3L, 2L, 
3L, 3L, 1L, 2L, 2L, 3L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 
1L, 2L, 3L, 1L, 3L, 1L, 1L, 1L, 2L, 2L, 1L, 2L, 2L, 3L, 2L, 3L, 
3L, 1L, 2L, 2L, 3L, 1L, 2L, 1L, 1L, 2L, 1L, 3L, 1L, 1L, 2L, 1L, 
3L, 3L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 3L, 2L, 3L, 3L, 2L, 2L, 1L, 
1L, 2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 1L, 2L, 3L, 1L, 3L, 3L, 1L, 
1L, 1L, 3L, 3L, 2L, 2L, 1L, 2L, 2L, 3L, 2L, 3L, 3L, 1L, 1L, 2L, 
2L, 3L, 2L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 1L, 3L, 1L, 1L, 2L, 3L, 
1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 3L, 2L, 2L, 1L, 2L, 2L, 3L, 2L, 
3L, 1L, 2L, 2L, 3L, 2L, 1L, 1L, 2L, 3L, 1L, 2L, 1L, 3L, 1L, 1L, 
3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 3L, 2L, 2L, 1L, 2L, 2L, 3L, 
2L, 3L, 3L, 1L, 2L, 2L, 3L, 2L, 1L, 1L, 2L, 3L, 1L, 1L, 2L, 1L, 
3L, 1L, 1L, 2L, 3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 3L, 2L, 2L, 1L, 
2L, 2L, 3L, 2L, 3L, 3L, 1L, 2L, 2L, 3L, 2L, 1L, 2L, 3L, 1L, 2L, 
1L, 3L, 1L, 1L, 3L, 1L, 3L, 3L, 3L, 1L, 1L, 1L, 2L, 2L, 1L, 2L, 
2L, 3L, 2L, 3L, 3L, 1L, 2L, 2L, 3L, 1L, 2L, 3L, 1L, 2L, 1L, 3L, 
1L, 1L, 3L, 1L, 3L, 3L, 3L), .Label = c("A", "B", "C"), class = "factor")), row.names = c(NA, 
-664L), class = c("tbl_df", "tbl", "data.frame"))

Upvotes: 0

Views: 126

Answers (1)

Ben Bolker
Ben Bolker

Reputation: 226247

This might be more appropriate for CrossValidated, as it has more to do with what kind of a model you ought to set up rather than how to do it, but here goes.

Let's work through your models.

result ~ school_type * subject + (1|year)

This says that the result depends on school type, subject, and their interaction (i.e. the differences between subjects vary among school types), and that the results vary as a random effect across years (i.e., you are only interested in the variability among years, not in making statistical comparisons between years).

issues:

  • this warns you that you have a "singular fit" — the among-year variability is estimated as zero. This probably occurs because you don't have many years in your sample, and is not necessarily fatal, but does indicate the model may be suboptimal (see e.g. here and here)
  • as you hint/suggest, this model does not account for the fact that you have multiple measurements per school. It assumes there is no variability among schools/no correlation within schools, which is almost certainly a problem.
result ~ school_type * subject + (year|school_ID)

In addition to the school type/subject interaction listed above, this model assumes that there is variation among schools and variation among years and that the differences in year effects across schools may be correlated (for example, schools that have a higher-than average effect in year 1 may on average also have higher-than-average effects in year 2).

  • this model is also singular
  • it is (nearly) the maximal model for this observational design; you could in theory also add (1|year) to allow for year-to-year variation that is consistent across schools (the current model only allows variation across schools of the year-to-year effect)
  • you said "nothing is significant". That's not what I see (and it's not a good way to figure out whether you have the right model or not! It's disappointing but entirely possible to have the correct model and have nothing be significant). Do you mean that the effect of school type is not significant (at the conventional p<=0.05) level?

For example, here are the results of car::Anova() applied to this fit:

Analysis of Deviance Table (Type II Wald chisquare tests)

Response: result
                        Chisq Df Pr(>Chisq)    
school_type            5.3273  2   0.069692 .  
subject             1030.9423  3  < 2.2e-16 ***
school_type:subject   22.1616  6   0.001132 ** 

Note that you should be very careful here in interpreting the tests of the main effects: read the "Details" section of ?car::Anova (and consider setting sum-to-zero contrasts if you are going to do type-3 tests).

In general, I would also suggest that you not look at significance tests until you have decided that you are happy with the structure of your model.

I'm going to suggest two more models:

result ~ school_type * subject + (1|year) + (1|school_ID) + (1|year:school_ID)

This allows for variation among years, among schools, and independent variation across years within schools. It's sensible (and more parsimonious than the maximal model above; it ignores the possible correlation of year-to-year variation across schools), but it still gives a singular model.

result ~ school_type * subject + year + (1|school_ID/year)

This changes the year effect from random to fixed, which is a reasonable strategy when you don't have very many levels with which to estimate a variance. (The (1|school_ID/year) term means "among-school variation and among-year variation nested within schools", and is identical to the combination of the other two random-effect terms in the previous model)

This model is not singular.

All of the models except the first (which neglects among-school variation) give nearly identical results for the school_type*subject part of the model, which is not really surprising since we're basically just slicing up the variation in (school × year) in slightly different ways.

Upvotes: 3

Related Questions