CODEWITHSUNDEEP
pythonpandasappend
sriramn
sriramn

Reputation: 2378

pandas: Convert Series of DataFrames to single DataFrame

I have a pandas Series object with each value being a DataFrame. I am trying convert this into a single DataFrame with all of the Series values (individual DataFrame) stacked on top of each other. How can I achieve this without a loop?

A toy example below to generate the test object (results). 

import pandas as pd
import numpy as np
numrows = 10000

def toy_function(x):
    silly_sequence = np.random.uniform(10, 100, (x+1))
    toy = pd.DataFrame({'ID':pd.Series(np.random.random_integers(1,20,3)),'VALUE':pd.Series((np.median(silly_sequence),np.mean(silly_sequence), np.max(silly_sequence)))})

    return toy

results = pd.DataFrame({'ID':range(numrows)})['ID'].apply(toy_function)

results is of Series type and each element is a DataFrame like so:

In [1]: results[1]
Out[1]: 
   ID      VALUE
0  17  40.035398
1   8  40.035398
2  20  66.483083

I am looking for a way to stack results[1], results[2] etc. on top of each other to yield a DataFrame like this:

   ID      VALUE
0  17  40.035398
1   8  40.035398
2  20  66.483083
4  12  25.035398
5   1  25.135398
6  19  65.553083
...

Upvotes: 19

Views: 7304

Answers (2)

jwilner
jwilner

Reputation: 6606

Try using pd.concat. At the very least, pd.concat(series.tolist()) should work.

Its default is to take a list of pandas dataframes or series and return them tacked end on end. http://pandas.pydata.org/pandas-docs/stable/merging.html

Upvotes: 27

Alexander
Alexander

Reputation: 109546

Concatenate results and ignore your index while doing so:

df_stacked = pd.concat([r for r in results], ignore_index=True)

Upvotes: 9

Related Questions