CODEWITHSUNDEEP
pythonpython-3.xpandasdatetimedataframe
user9996043
user9996043

Reputation: 229

Delete rows with dates before the required date point based on key value

I have a pd.dataframe that looks like this:

key_value     date
value_01   2017-01-13
value_01   2018-02-17
value_01   2018-04-02
value_01   2018-05-13
value_01   2018-05-16  
value_02   2017-01-18
value_02   2018-03-13
value_02   2018-04-01
value_02   2018-05-16  
value_02   2018-05-22  
value_03   2018-01-13
value_03   2018-04-14

So now based on the key_value,

I want to drop all the rows that have their date column value before 2018-04-01

I want to have an end output like this:

 key_value     date
value_01   2018-04-02
value_01   2018-05-13
value_01   2018-05-16  
value_02   2018-04-01
value_02   2018-05-16  
value_02   2018-05-22  
value_03   2018-04-14

Upvotes: 9

Views: 41512

Answers (4)

Jayron Soares
Jayron Soares

Reputation: 461

for future bypassers, here is a simple solution using an index.

idx_names = df_bulbr[df_bulbr.disconnected_time =='1970-01-01 00:00:00'].index
df_bulbr.drop(idx_names, inplace=True)
df_bulbr.reset_index()

Upvotes: 1

BananaJoe
BananaJoe

Reputation: 45

A little bit late but here is my solution. Tried to use some pythonic stuff without pandas. Maybe more easy to read.

from datetime import datetime

data = {}
specificDate = datetime.strptime("2018-04-01", "%Y-%m-%d")

data.update({"value_01" : ["2017-01-13", "2018-02-17", "2018-04-02", "2018-05-13", "2018-05-16"]})
data.update({"value_02" : ["2017-01-18", "2018-03-13", "2018-04-01", "2018-05-16", "2018-05-22"]})
data.update({"value_03" : ["2018-01-13", "2018-04-14"]})

for key in data.keys():
    data.update({key : list(filter(lambda x: datetime.strptime(x, "%Y-%m-%d") >= specificDate ,data[key]))})

for key, value in data.items():
    print(key)
    for val in value:
        print("    " + val)

Output:

value_01
    2018-04-02
    2018-05-13
    2018-05-16
value_02
    2018-04-01
    2018-05-16
    2018-05-22
value_03
    2018-04-14

Upvotes: 0

Carlo 1585
Carlo 1585

Reputation: 1477

Maybe this code is not the best but does what you asked even if your date are not sorted.

import pandas as pd
from datetime import datetime

d = {'key_value': [1, 2, 3, 4, 5], 'date': ['2017-01-13', '2018-02-17','2018-04-02','2018-05-13','2018-05-16']}#create dataframe

date_string='2018-04-01'#date limit
date_to_drop=datetime.strptime(date_string, '%Y-%m-%d')# conmert my date to datetime
i=0
l=len(d['date'])#len of your set of date
while i<l:#loop on your set of date
    datetime_object = datetime.strptime(d['date'][i], '%Y-%m-%d')#convert the current date in datetime
    if datetime_object<date_to_drop:#if my current date is previous of the date limit I delete it from my dataframe
        d['date'].pop(i)#delete the date
        d['key_value'].pop(i)#delete the key_value
        l-=1#decrese the len of the date set of 1 seeing that I delete an element
    else:#if my current date is after of date limit I just pass to next iteration
        i+=1
df = pd.DataFrame(data=d)
print (df)

This is the result

         date  key_value
0  2018-04-02          3
1  2018-05-13          4
2  2018-05-16          5

Upvotes: 0

jpp
jpp

Reputation: 164623

You can just filter your dataframe using Boolean indexing. There is no groupwise operation here. Just remember to convert your series to datetime first.

df['date'] = pd.to_datetime(df['date'])

res = df[~(df['date'] < '2018-04-01')]

print(res)

   key_value       date
2   value_01 2018-04-02
3   value_01 2018-05-13
4   value_01 2018-05-16
7   value_02 2018-04-01
8   value_02 2018-05-16
9   value_02 2018-05-22
11  value_03 2018-04-14

Upvotes: 21

Related Questions