CODEWITHSUNDEEP
pythonpandas
user55641
user55641

Reputation: 45

pandas .loc returns empty dataframe

I have pandas dataframe, which looks like below. 

chainage(km)  
0  
0.001  
0.002  
0.003  
0.004

while I use .loc to search for the chainage(km) it returns the empty dataframe for some chainages.

print data.loc[data['chainage(km)'] == float(0.004)]

-- Empty dataframe 

print data.loc[data['chainage(km)'] == float(0.001)]

-- returns value

Any help would be much appreciated.

Upvotes: 3

Views: 2085

Answers (2)

Haleemur Ali
Haleemur Ali

Reputation: 28253

Two possible causes:

It is possible that the column chainage(km) is an object type, and for the fifth row it stores 0.004 as a string, i.e. '0.004'. To fix that cast it as float

data = data.astype(float)

If after doing the above casting, the filtering still doesn't work for 0.004

Then, given that data['chainage(km)'] == float(0.004) doesn't return any True, while data.loc[data['chainage(km)'] == float(0.001)] returns correctly, it suggest that you are experiencing floating point errors.

Try the following:

x = float(0.004)
data[abs(data['chainage(km)'] - float(x)) < 0.0001*x]

This filters for the row replacing the equality condition with an arbitrarily sized error.

Upvotes: 0

cs95
cs95

Reputation: 402523

The problem arises due to floating point inaccuracies. This is explained in Is floating point math broken?.

In situations like this, please use np.isclose instead.

df[np.isclose(data['chainage(km)'], 0.004)]

Upvotes: 3

Related Questions