CODEWITHSUNDEEP
pythonpandasdataframedatetime64
Aleminco
Aleminco

Reputation: 45

How to merge a list containing data and datetime64[ns] with a pandas dataframe with datetime64[ns] index

I want to read two columns S1_max and S2_max from a dataframe data. Wherever a value is present in the S1_max column I want to check that each S1_max is succeeded by a corresponding S2_max signal. If so I calculate the time delta between the S1_max and S2_max signal. This result is then indexed at the datetime[64ns] index of the S2_max column in a separate dict dwhich is then appended to a list delta_data . How can I add this result to my already existing data dataframe at the corresponding datetime[64ns] index?

This is my creation of delta_data:

#time between each S2 global maxima: 86 ns/samp freq 200 = 0.43 ns
#Checking that each S1 is succeeded by a corresponging S2 signal and calculating the time delta:
delta_data = []
diff_S1 = 0
diff_S2 = 0
i = 0
while((i + diff_S1 + 1 < len(peak_indexes_S1)) and (i + diff_S2<len(peak_indexes_S2))):
# Find next ppg peak after S1 peak
    while (df["S2"].index[peak_indexes_S2[i + diff_S2]] < df["S1"].index[peak_indexes_S1[i+diff_S1]]):
        diff_S2=diff_S2+1

    while (df["S1"].index[peak_indexes_S1[i+diff_S1+1]] < df["S2"].index[peak_indexes_S2[i + diff_S2]]):
        diff_S1=diff_S1+1

    i_peak_S2 = peak_indexes_S2[i + diff_S2]
    i_peak_S1 = peak_indexes_S1[i + diff_S1]

    d={}
    d["td"] = (df["S2"].index[i_peak_S2]-df["S1"].index[i_peak_S1]).microseconds
    d["time"] = df["S2"].index[i_peak_S2]
    PATdata.append(d)

    i = i + 1

time_delta=pd.DataFrame(delta_data)

delta_data printed out:

         td                    time
0    355000 2019-08-07 13:06:31.010
1    355000 2019-08-07 13:06:31.850
2    355000 2019-08-07 13:06:32.695

This is my data dataframe:

                           l1        l2        l3        l4       S1       S2   S2_max   S1_max

2019-08-07 13:11:21.485  0.572720  0.353433  0.701320  1.418840  4.939690  2.858326  2.858326       NaN
2019-08-07 13:11:21.490  0.572807  0.353526  0.701593  1.419052  4.939804  2.854604       NaN  4.939804

This dataframe is created by:

data = pd.read_csv('file.txt')
data.columns = ['l1','l2','l3','l4','S1','S2']
nbrMeasurments = sum(1 for line in open('file.txt'))
data.index = pd.date_range('2019-08-07 13:06:30'), periods=nbrMeasurments-1, freq="5L")

I have tried DataFrame.combine_firstand append.

Also, the same problem occurs when trying to add another dataframe to data. This dataframe doesn't have ms in the datetime frame:

                     S3   S4 
Date                                       
2019-08-07 13:06:30         111          61

Upvotes: 1

Views: 208

Answers (1)

JacoSolari
JacoSolari

Reputation: 1394

As far as I could understand you are trying to append another column to an existing DataFrame.

here how to do it:

df1 = pd.DataFrame({'names':['bla', 'blah', 'blahh'], 'values':[1,2,3]})
df2_to_concat = pd.DataFrame({'put_me_as_a_new_column':['row1', 'row2', 'row3']})

pd.concat([df1.reset_index(drop=True), df2_to_concat.reset_index(drop=True)], axis=1)

The reset_index(drop=True) makes sure you don't produce NaNs or duplicate index columns.

Upvotes: 0

Related Questions