Reputation: 803
The df looks like below:
Time A
2019-05-18 01:15:28 7
2019-05-18 01:28:11 7
2019-05-18 01:36:36 12
2019-05-18 01:39:47 12
2019-05-18 01:53:32 12
2019-05-18 02:05:37 7
I understand how to calculate consecutive row time difference. But I want to calculate the time difference whenever the Value in A is 7 to 12.
Expected output:
Time A Time_difference
2019-05-18 01:15:28 7 0
2019-05-18 01:28:11 7 0
2019-05-18 01:36:36 12 00:21:08
2019-05-18 01:39:47 12 0
2019-05-18 01:53:32 12 0
2019-05-18 02:05:37 12 0
Upvotes: 4
Views: 697
Reputation: 5481
Explanation:
import pandas as pd
import numpy as np
np.random.seed(10)
date_range = pd.date_range("25-9-2019", "27-9-2019", freq="3H")
df = pd.DataFrame({'Time':date_range, 'A':np.random.choice([5,7,12], len(date_range))})
df["Seven"] = (df["A"] == 7).cumsum()
# display(df)
pass_to_next_group = {"val": None}
def diff(group):
group["Diff"]=0
loc = group.index[group["A"]==12]
time_a = pass_to_next_group["val"] if pass_to_next_group["val"] else group["Time"].iloc[0]
pass_to_next_group["val"] = None
if group.name>0 and len(loc)>0:
group.loc[loc[0],"Diff"] = time_a-group.loc[loc[0],"Time"]
else:
pass_to_next_group["val"] = time_a
return group
df.groupby("Seven").apply(diff)
Upvotes: 1
Reputation: 863791
Sample:
times = [
'2019-05-18 01:15:28',
'2019-05-18 01:28:11',
'2019-05-18 01:36:36',
'2019-05-18 01:39:47',
'2019-05-18 01:53:32',
'2019-05-18 02:05:37'
]
a = [7, 7, 12, 7, 12, 7]
df = pd.DataFrame({'times': pd.to_datetime(times), 'A':a})
print (df)
times A
0 2019-05-18 01:15:28 7
1 2019-05-18 01:28:11 7
2 2019-05-18 01:36:36 12
3 2019-05-18 01:39:47 7
4 2019-05-18 01:53:32 12
5 2019-05-18 02:05:37 7
First create default index and filter rows with 7
and 12
only:
df = df.reset_index(drop=True)
df1 = df[df['A'].isin([7, 12])]
Then get first consecutive values in rows with compare with shifted values:
df1 = df1[df1['A'].ne(df1['A'].shift())]
print (df1)
times A
0 2019-05-18 01:15:28 7
2 2019-05-18 01:36:36 12
3 2019-05-18 01:39:47 7
4 2019-05-18 01:53:32 12
5 2019-05-18 02:05:37 7
Then filter 7
with next 12
rows:
m1 = df1['A'].eq(7) & df1['A'].shift(-1).eq(12)
m2 = df1['A'].eq(12) & df1['A'].shift().eq(7)
df2 = df1[m1 | m2]
print (df2)
times A
0 2019-05-18 01:15:28 7
2 2019-05-18 01:36:36 12
3 2019-05-18 01:39:47 7
4 2019-05-18 01:53:32 12
Get datetimes with pair and unpairs rows:
out7 = df2.iloc[::2]
out12 = df2.iloc[1::2]
And last subtract:
df['Time_difference'] = out12['times'] - out7['times'].to_numpy()
df['Time_difference'] = df['Time_difference'].fillna(pd.Timedelta(0))
print (df)
times A Time_difference
0 2019-05-18 01:15:28 7 00:00:00
1 2019-05-18 01:28:11 7 00:00:00
2 2019-05-18 01:36:36 12 00:21:08
3 2019-05-18 01:39:47 7 00:00:00
4 2019-05-18 01:53:32 12 00:13:45
5 2019-05-18 02:05:37 7 00:00:00
Upvotes: 1
Reputation: 2629
You can isolate any values in dataframes using loc
. What gets returned is a Series, which can be indexed like a list. Use [0]
to get the first occurrence in the Series.
times = [
'2019-05-18 01:15:28',
'2019-05-18 01:28:11',
'2019-05-18 01:36:36',
'2019-05-18 01:39:47',
'2019-05-18 01:53:32',
'2019-05-18 02:05:37'
]
a = [9, 7, 7, 5, 12, 12]
df = pd.DataFrame({'times':times, 'a':a})
df.times = pd.to_datetime(df['times'])
pd.Timedelta(df.loc[df.a == 12, 'times'].values[0] - df.loc[df.a == 7, 'times'].values[0])
Timedelta('0 days 00:25:21')
Or we can break that code apart for readability's sake and do the calculations on new variables:
times = [
'2019-05-18 01:15:28',
'2019-05-18 01:28:11',
'2019-05-18 01:36:36',
'2019-05-18 01:39:47',
'2019-05-18 01:53:32',
'2019-05-18 02:05:37'
]
a = [9, 7, 7, 5, 12, 12]
df = pd.DataFrame({'times':times, 'a':a})
df.times = pd.to_datetime(df['times'])
end = df.loc[df.a == 12, 'times'].values[0]
start = df.loc[df.a == 7, 'times'].values[0]
pd.Timedelta(end - start)
Timedelta('0 days 00:25:21')
Upvotes: 2