CODEWITHSUNDEEP
pythonpandasdatetime
Jack Harding
Jack Harding

Reputation: 87

Comparing datetime object to Panda series elements

I'm trying to compare a datetime object to a date stored in a Panda Series. For every element in the Series that matches the datetime object passed, that element is appended to an array. The demand is a numpyfloat64. 

date_chosen = dt.datetime(2019, 4, 2) 
raw_csv = pd.read_csv(data_series, sep=',', na_values=missing_values)

demand_s = pd.to_numeric(raw_csv['DEMAND'])          # extracts demand
date_series = pd.to_datetime(raw_csv['DATE'])        # extracts date

demand_needed = []                        # which demand values match the date_chosen
day = date_series.dt.day                  # only includes day 
for i in day:
    if day[i] == date_chosen.day:         # if element in day is same as chosen one
        demand_needed.append(demand_s[i]) # append matching element 

print(type(date_chosen.day))              # = int
print(type(day[2]))                       # = numpy.int64

This runs fine but the issue is the demand_needed[] is empty. The date_chosen.day is a standard int and elements of day are numpyint64. How can I compare int and numpyint64?

Upvotes: 3

Views: 65

Answers (1)

Chris Adams
Chris Adams

Reputation: 18647

In your for loop, i is the value of each row in the Series "day", it's not the index. So your loop should be structured more like:

date_chosen = dt.datetime(2019, 4, 2) 
raw_csv = pd.read_csv(data_series, sep=',', na_values=missing_values)

demand_s = pd.to_numeric(raw_csv['DEMAND'])
date_series = pd.to_datetime(raw_csv['DATE'])

demand_needed = []
day = date_series.dt.day
for idx, d in day.iteritems():
    if d == date_chosen.day:
        demand_needed.append(demand_s.iloc[idx])

But a better solution IIUC, would be to use boolean indexing rather than iterating:

demand_needed = raw_csv.loc[raw_csv.DATE.dt.day.eq(date_chosen.day), 'DEMAND']

Or if you need the output as a list instead of Series, use:

demand_needed = raw_csv.loc[raw_csv.DATE.dt.day.eq(date_chosen.day), 'DEMAND'].tolist()

Upvotes: 4

Related Questions