CODEWITHSUNDEEP
pythonpandas
Carlos Cariello
Carlos Cariello

Reputation: 163

Detect sign changes in Pandas Dataframe

I have a pandas dataframe that is datetime indexed and it looks like this:

Datetime
2020-05-11 14:00:00-03:00    0.097538
2020-05-11 14:30:00-03:00   -0.083788
2020-05-11 15:00:00-03:00   -0.074128
2020-05-11 15:30:00-03:00    0.059725
2020-05-11 16:00:00-03:00    0.041369
2020-05-11 16:30:00-03:00    0.034388
2020-05-12 10:00:00-03:00    0.006814
2020-05-12 10:30:00-03:00   -0.005308
2020-05-12 11:00:00-03:00   -0.036952
2020-05-12 11:30:00-03:00   -0.070307
2020-05-12 12:00:00-03:00    0.102004
2020-05-12 12:30:00-03:00   -0.139317
2020-05-12 13:00:00-03:00   -0.167589
2020-05-12 13:30:00-03:00   -0.179942
2020-05-12 14:00:00-03:00    0.182351
2020-05-12 14:30:00-03:00   -0.160736
2020-05-12 15:00:00-03:00   -0.150033
2020-05-12 15:30:00-03:00   -0.141862
2020-05-12 16:00:00-03:00   -0.121372
2020-05-12 16:30:00-03:00   -0.095990
Name: result_col, dtype: float64

My need is to mark the rows where it changes signal, from negative to positive and vice-versa. Any thoughts on how to achieve it?

Edit: I need +1 on the cross up and -1 on the cross down.

Upvotes: 9

Views: 6566

Answers (5)

Lukas
Lukas

Reputation: 2312

When someone needs to get just the change rows in a dataframe, then usage of @BENY or @Marat solution is great. In my case I need to get indices where row values change from 0 to 1 and back:

import pandas as pd
df = pd.DataFrame({'signal': [1,0,0,0,1,1,1,0,0,1,1,1,1,0,0]})

df.loc[(np.sign(df['signal']).diff().ne(0))]

Result:

    signal
0   1
1   0
4   1
7   0
9   1
13  0

My motivation is to draw rectangle in plotly graphs when signal is 1 and finish the rectangle when 0 comes

Upvotes: 2

Dr Acme Isme
Dr Acme Isme

Reputation: 55

Here is a pandas specific solution, using list comprehension and a function call to speed things up a little. If someone has an entirely vectorized solution that would be best, but as for now this solution should be pretty snappy:

def compare_sign(s):
    if len(s) == 2:
        return s.iat[1] if s.iat[0] != s.iat[1] else 0
    else:
        return 0  # this case covers the first value in the rolling window series

def cross(df, field):
    return pd.Series([compare_sign(w) for w in df[field].apply(np.sign).rolling(2)])

This solution returns 0 when the value hits zero without crossing. so for example: [-2, -1, 0, -1] -> [0, 0, 0, -1] For different behavior just tweak the compare_sign function.

Upvotes: 0

Marat
Marat

Reputation: 15738

# assuming series is called 'data'
sign = data > 0
sign_change = (sign != sign.shift(1))  # or -1, depending if you want True before the sign change

UPD: I think BEN_YO's solution is better. I wasn't aware of .diff at the time of the answer

Upvotes: 6

Diggy.
Diggy.

Reputation: 6944

You could iterate through the column, keeping track of the current and previous index's values. The logic could be similar to:

prev_value = 0
sign_changes = []
for i, value in enumerate(column, 0): # starting index of 0
    if value > 0 and prev_value < 0 or value < 0 and prev_value > 0:
        sign_changes.append([i-1, i])

Upvotes: 1

BENY
BENY

Reputation: 323226

Let us try 

import numpy as np 
np.sign(data).diff().ne(0)

Upvotes: 15

Related Questions