CODEWITHSUNDEEP
pythonpandasdataframeloops
GTek
GTek

Reputation: 155

Iterating over multiple dataframes

##1    
M_members = [1000 , 1450, 1900]  
M = pd.DataFrame(M_members)

##2
a_h_members = [0.4 , 0.6 , 0.8 ]
a_h = pd.DataFrame(a_h_members) 

##3
d_h_members = [0.1 , 0.2 ]
d_h = pd.DataFrame(d_h_members)

As the output I want is in dataframe form:

1000 0.4 0.1
1000 0.4 0.2
1000 0.6 0.1
1000 0.6 0.2
1000 0.8 0.1
1000 0.8 0.2
1450 0.4 0.1
1450 0.4 0.2
1450 0.6 0.1
1450 0.6 0.2
1450 0.8 0.1
1450 0.8 0.2
1900 0.4 0.1
1900 0.4 0.2
1900 0.6 0.1
1900 0.6 0.2
1900 0.8 0.1
1900 0.8 0.2

I want to do this loop for more dataframes actually.

Upvotes: 3

Views: 116

Answers (4)

Soudipta Dutta
Soudipta Dutta

Reputation: 2162

import pandas as pd
from sklearn.model_selection import ParameterGrid

# Define the lists
M_members = [1000, 1450, 1900]
a_h_members = [0.4, 0.6, 0.8]
d_h_members = [0.1, 0.2]

# Create a dictionary for the parameter grid
param_grid = {
    'M': M_members,
    'a_h': a_h_members,
    'd_h': d_h_members
}

# Generate all combinations of the given arrays using ParameterGrid
combinations = list(ParameterGrid(param_grid))

# Convert the combinations to a DataFrame
df_combinations = pd.DataFrame(combinations)

print(df_combinations)
"""
     M  a_h  d_h
0   1000  0.4  0.1
1   1000  0.4  0.2
2   1000  0.6  0.1
3   1000  0.6  0.2
4   1000  0.8  0.1
5   1000  0.8  0.2
6   1450  0.4  0.1
7   1450  0.4  0.2
8   1450  0.6  0.1
9   1450  0.6  0.2
10  1450  0.8  0.1
11  1450  0.8  0.2
12  1900  0.4  0.1
13  1900  0.4  0.2
14  1900  0.6  0.1
15  1900  0.6  0.2
16  1900  0.8  0.1
17  1900  0.8  0.2

"""

Upvotes: 0

sammywemmy
sammywemmy

Reputation: 28729

Depending on your data size, expand_grid from pyjanitor may help with performance:

# pip install pyjanitor
import janitor as jn
import pandas as pd

others = {'a':M, 'b':a_h, 'c':d_h}
jn.expand_grid(others = others)

       a    b    c
       0    0    0
0   1000  0.4  0.1
1   1000  0.4  0.2
2   1000  0.6  0.1
3   1000  0.6  0.2
4   1000  0.8  0.1
5   1000  0.8  0.2
6   1450  0.4  0.1
7   1450  0.4  0.2
8   1450  0.6  0.1
9   1450  0.6  0.2
10  1450  0.8  0.1
11  1450  0.8  0.2
12  1900  0.4  0.1
13  1900  0.4  0.2
14  1900  0.6  0.1
15  1900  0.6  0.2
16  1900  0.8  0.1
17  1900  0.8  0.2

You can drop a column level, or flatten it:

jn.expand_grid(others = others).droplevel(axis = 1, level = 1)

       a    b    c
0   1000  0.4  0.1
1   1000  0.4  0.2
2   1000  0.6  0.1
3   1000  0.6  0.2
4   1000  0.8  0.1
5   1000  0.8  0.2
6   1450  0.4  0.1
7   1450  0.4  0.2
8   1450  0.6  0.1
9   1450  0.6  0.2
10  1450  0.8  0.1
11  1450  0.8  0.2
12  1900  0.4  0.1
13  1900  0.4  0.2
14  1900  0.6  0.1
15  1900  0.6  0.2
16  1900  0.8  0.1
17  1900  0.8  0.2

Upvotes: 1

mozway
mozway

Reputation: 262484

If you're starting from the DataFrames, you can use a repeated cross merge:

dfs = [M, a_h, d_h]

from functools import reduce

out = (reduce(lambda a,b: a.merge(b, how='cross'), dfs)
      .set_axis(range(len(dfs)), axis=1)
      )

Output:

       0    1    2
0   1000  0.4  0.1
1   1000  0.4  0.2
2   1000  0.6  0.1
3   1000  0.6  0.2
4   1000  0.8  0.1
5   1000  0.8  0.2
6   1450  0.4  0.1
7   1450  0.4  0.2
8   1450  0.6  0.1
9   1450  0.6  0.2
10  1450  0.8  0.1
11  1450  0.8  0.2
12  1900  0.4  0.1
13  1900  0.4  0.2
14  1900  0.6  0.1
15  1900  0.6  0.2
16  1900  0.8  0.1
17  1900  0.8  0.2

Upvotes: 0

rafaelc
rafaelc

Reputation: 59304

Use itertools.product

>>> import itertools
>>> pd.DataFrame(itertools.product(*[M_members, a_h_members, d_h_members]))

       0    1    2
0   1000  0.4  0.1
1   1000  0.4  0.2
2   1000  0.6  0.1
3   1000  0.6  0.2
4   1000  0.8  0.1
5   1000  0.8  0.2
6   1450  0.4  0.1
7   1450  0.4  0.2
8   1450  0.6  0.1
9   1450  0.6  0.2
10  1450  0.8  0.1
11  1450  0.8  0.2
12  1900  0.4  0.1
13  1900  0.4  0.2
14  1900  0.6  0.1
15  1900  0.6  0.2
16  1900  0.8  0.1
17  1900  0.8  0.2

Upvotes: 3

Related Questions