CODEWITHSUNDEEP
pythonpandasdataframe
blahblahblah
blahblahblah

Reputation: 2437

Sum a list of Pandas DataFrames

Is there a way to sum multiple pandas DataFrames using syntax similar to pd.concat([df1, df2, df3, df4]). I understand from documentation that I can do df1.sum(df2, fill_value=0), but I have a long list of DataFrames I need to sum and was wondering if I could do it without writing a loop.

Somewhat related question/answer: Pandas sum multiple dataframes (Stack Overflow)

Example of what the result should look like:

idx1 = pd.MultiIndex.from_tuples([('a', 'A'), ('a', 'B'), ('b', 'A'), ('b', 'D')])
idx2 = pd.MultiIndex.from_tuples([('a', 'A'), ('a', 'C'), ('b', 'A'), ('b', 'C')])
idx3 = pd.MultiIndex.from_tuples([('a', 'A'), ('a', 'D'), ('b', 'A'), ('b', 'C')])

np.random.seed([3,1415])
df1 = pd.DataFrame(np.random.randn(4, 1), idx1, ['val'])
df2 = pd.DataFrame(np.random.randn(4, 1), idx2, ['val'])
df3 = pd.DataFrame(np.random.randn(4, 1), idx3, ['val'])

df1

enter image description here

df2

enter image description here

df3

enter image description here

The result should look like:

enter image description here

Upvotes: 18

Views: 19175

Answers (1)

jezrael
jezrael

Reputation: 862511

Use reduce with add with parameter fill_value=0:

np.random.seed(12)

a = pd.DataFrame(np.random.randint(3, size=(5,3)), columns=list('abc'))
b = pd.DataFrame(np.random.randint(3, size=(5,2)), columns=list('ab'))
c = pd.DataFrame(np.random.randint(3, size=(5,2)), columns=list('ac'))
print(a)
   a  b  c
0  2  1  1
1  2  0  0
2  2  1  0
3  1  1  1
4  2  2  2

print(b)
   a  b
0  0  1
1  0  0
2  1  2
3  1  2
4  0  1

print(c)
   a  c
0  2  0
1  2  2
2  2  0
3  0  2
4  1  1

from functools import reduce

dfs = [a,b, c]
d = reduce(lambda x, y: x.add(y, fill_value=0), dfs)
print (d)
   a    b    c
0  4  2.0  1.0
1  4  0.0  2.0
2  5  3.0  0.0
3  2  3.0  3.0
4  3  3.0  3.0

Upvotes: 39

Related Questions