CODEWITHSUNDEEP
pythonpandasapply
RSHAP
RSHAP

Reputation: 2446

Pandas sequentially apply function using output of previous value

I want to compute the "carryover" of a series. This computes a value for each row and then adds it to the previously computed value (for the previous row).

How do I do this in pandas?

decay = 0.5
test = pd.DataFrame(np.random.randint(1,10,12),columns = ['val'])
test
    val
0   4
1   5
2   7
3   9
4   1
5   1
6   8
7   7
8   3
9   9
10  7
11  2

decayed = []
for i, v in test.iterrows():
    if i ==0:
        decayed.append(v.val)
        continue
    d = decayed[i-1] + v.val*decay
    decayed.append(d)

test['loop_decay'] = decayed
test.head()

    val loop_decay
0   4   4.0
1   5   6.5
2   7   10.0
3   9   14.5
4   1   15.0

Upvotes: 0

Views: 2905

Answers (2)

Brian Huey
Brian Huey

Reputation: 1630

You can utilize pd.Series.shift() to create a dataframe with val[i] and val[i-1] and then apply your function across a single axis (1 in this case):

 # Create a series that shifts the rows by 1
 test['val2'] = test.val.shift()
 # Set the first row on the shifted series to 0
 test['val2'].ix[0] = 0
 # Apply the decay formula:
 test['loop_decay'] = test.apply(lambda x: x['val'] + x['val2'] * 0.5, axis=1)

Upvotes: 1

Parfait
Parfait

Reputation: 107652

Consider a vectorized version with cumsum() where you cumulatively sum (val * decay) with the very first val.

However, you then need to subtract the very first (val * decay) since cumsum() includes it:

test['loop_decay'] = (test.ix[0,'val']) + (test['val']*decay).cumsum() - (test.ix[0,'val']*decay)

Upvotes: 2

Related Questions