CODEWITHSUNDEEP
pandasgroup-byseriest-test
tester777
tester777

Reputation: 13

Group by groups to Pandas Series/Dataframe

Apologies in advance if this question is naive. I am new to Python. I am trying to perform a t-test on two columns of my dataframe. It only makes sense to do the t-test after having grouped the columns by another column in the same dataframe.

I am dealing with something like this:

rand_array = np.random.randint(low=10, high=30, size=9)
rand_array2 = np.random.randint(low=10, high=30, size=9)
d = {'key1':[0,0,1,0,1,1,1,0,1], 'key2': rand_array, 'key3': rand_array2}
df1 = pd.DataFrame(d)
print df1

The output I get is:

     key1  key2  key3
0     0    20    18
1     0    22    16
2     1    21    26
3     0    21    13
4     1    11    21
5     1    23    10
6     1    17    29
7     0    13    25
8     1    24    29

Then, I group by key1

g1 = df1.groupby('key1')
print g1.groups
>>> {0: Int64Index([0, 1, 3, 7], dtype='int64'), 1: Int64Index([2, 4, 5, 6, 8], dtype='int64')}

I want to perform a t-test on basically 0: Int64Index([0, 1, 3, 7], dtype='int64') vs 1: Int64Index([2, 4, 5, 6, 8], dtype='int64').

Is this possible?

Thank you!

Upvotes: 1

Views: 385

Answers (1)

piRSquared
piRSquared

Reputation: 294218

See Welch's T-Test

I'd do it like this:

def welch_ttest(x1, x2):
    x_1 = x1.mean()
    x_2 = x2.mean()
    s1 = x1.std()
    s2 = x2.std()
    n1 = len(x1)
    n2 = len(x2)
    return ((x_1 - x_2) / (np.sqrt(s1 ** 2 / n1 + s2 ** 2 / n2)))

def grouped_welch_ttest(df):
    return welch_ttest(df.key2, df.key3)

df1.groupby('key1').apply(grouped_welch_ttest)

key1
0   -1.471497
1    1.487045
dtype: float64

Upvotes: 2

Related Questions