CODEWITHSUNDEEP
pythonpandasdataframesum
Zesima29
Zesima29

Reputation: 184

Summing by string names Pandas

I'm working with a data frame like this, but bigger and with more zone. I am trying to sum the value of the rows by their names. The total sum of the R or C zones goes in total column while the total sum of either M zones goes in total1 .

Input:

total, total1 are the desired output.

ID  Zone1   CHC1    Value1  Zone2     CHC2  Value2  Zone3   CHC3    Value3  total   total1
 1  R5B     100      10       C2        0     20      R10A   2       5        35       0
 1  C2       95      20      M2-6       5      6      R5B    7       3        23       6       
 3  C2       40      4        C4       60      6       0     6       0        10       0
 3  C1       100     8         0        0      0       0    100      0        8        0
 5  M1-5     10      6       M2-6      86     15       0     0       0        0        21

Upvotes: 1

Views: 137

Answers (2)

gyoza
gyoza

Reputation: 2152

Solution1 (simpler code but slower and less flexible) 

total = []
total1 = []

for i in range(df.shape[0]):
    temp = df.iloc[i].tolist()
    if "R2" in temp:
        total.append(temp[temp.index("R2")+1])
    else:
        total.append(0)
    if ("C1" in temp) & ("C4" in temp):
        total1.append(temp[temp.index("C1")+1] + temp[temp.index("C4")+1])
    else:
        total1.append(0)

df["Total"] = total
df["Total1"] = total1

Solution2 (faster than solution1 and easier to customize but possibly memory intensive)

# columns to use
cols = df.columns.tolist()
zones = [x for x in cols if x.startswith('Zone')]
vals = [x for x in cols if x.startswith('Value')]

# you can customize here
bucket1 = ['R2']
bucket2 = ['C1', 'C4']
thresh = 2 # "OR": 1, "AND": 2

original = df.copy()

# bucket1 check
for zone in zones:
    df.loc[~df[zone].isin(bucket1), cols[cols.index(zone)+1]] = 0

original['Total'] = df[vals].sum(axis=1)
df = original.copy()

# bucket2 check
for zone in zones:
    df.loc[~df[zone].isin(bucket2), cols[cols.index(zone)+1]] = 0

df['Check_Bucket'] = df[zones].stack().reset_index().groupby('level_0')[0].apply(list)
df['Check_Bucket'] = df['Check_Bucket'].apply(lambda x: len([y for y in x if y in bucket2]))
df['Total1'] = df[vals].sum(axis=1)
df.loc[df.Check_Bucket < thresh, 'Total1'] = 0
df.drop('Check_Bucket', axis=1, inplace=True)

When I expanded original dataframe to 100k rows, solution 1 took 11.4 s ± 82.1 ms per loop, while solution 2 took 3.53 s ± 29.8 ms per loop. The difference is because solution 2 does not for-looping over row direction.

Upvotes: 1

jezrael
jezrael

Reputation: 862701

You can use filter for DataFrames for Zones and Values:

z = df.filter(like='Zone')
v = df.filter(like='Value')

Then create boolean DataFrames by contains with apply if want check substrings:

m1 = z.apply(lambda x: x.str.contains('R|C'))
m2 = z.apply(lambda x: x.str.contains('M'))

#for check strings
#m1 = z == 'R2'
#m2 = z.isin(['C1', 'C4'])

Last filter by where v and sum per rows:

df['t'] = v.where(m1.values).sum(axis=1).astype(int)
df['t1'] = v.where(m2.values).sum(axis=1).astype(int)

print (df)
   ID Zone1  CHC1  Value1 Zone2  CHC2  Value2 Zone3  CHC3  Value3   t  t1
0   1   R5B   100      10    C2     0      20  R10A     2       5  35   0
1   1    C2    95      20  M2-6     5       6   R5B     7       3  23   6
2   3    C2    40       4    C4    60       6     0     6       0  10   0
3   3    C1   100       8     0     0       0     0   100       0   8   0
4   5  M1-5    10       6  M2-6    86      15     0     0       0   0  21

Upvotes: 3

Related Questions