CODEWITHSUNDEEP
python-2.7pandastime-seriesinterpolation
jbbj94
jbbj94

Reputation: 334

Pandas Interpolate Returning NaNs

I'm trying to do basic interpolation of position data at 60hz (~16ms) intervals. When I try to use pandas 0.14 interpolation over the dataframe, it tells me I only have NaNs in my data set (not true). When I try to run it over individual series pulled from the dataframe, it returns the same series without the NaNs filled in. I've tried setting the indices to integers, using different methods, fiddling with the axis and limit parameters of the interpolation function - no dice. What am I doing wrong?

df.head(5) : 
         x        y        ms
0  20.5815  14.1821  333.3333
1      NaN      NaN       350
2  20.6112  14.2013  366.6667
3      NaN      NaN  383.3333
4  20.5349  14.2232       400


df = df.set_index(df.ms) # set indices to milliseconds

When I try running 

df.interpolate(method='values')

I get this error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-462-cb0f1f01eb84> in <module>()
     12 
     13 
---> 14 df.interpolate(method='values')
     15 
     16 

/Users/jsb/anaconda/lib/python2.7/site-packages/pandas/core/generic.pyc in interpolate(self, method, axis, limit, inplace, downcast, **kwargs)
   2511 
   2512         if self._data.get_dtype_counts().get('object') == len(self.T):
-> 2513             raise TypeError("Cannot interpolate with all NaNs.")
   2514 
   2515         # create/use the index

TypeError: Cannot interpolate with all NaNs.

I've also tried running over individual series, which only return what I put in:

temp = df.x
temp.interpolate(method='values')

333.333333    20.5815
350.000000        NaN
366.666667    20.6112
383.333333        NaN
400.000000    20.5349 Name: x, dtype: object

EDIT :

Props to Jeff for inspiring the solution.

Adding:

df[['x','y','ms']] = df[['x','y','ms']].astype(float)

before

df.interpolate(method='values')

interpolation did the trick.

Upvotes: 3

Views: 5165

Answers (2)

lukewitmer
lukewitmer

Reputation: 1183

Based on your edit with props to Jeff for inspiring the solution.

Adding:

df = df.astype(float)

before

df.interpolate(method='values')

interpolation did the trick for me as well. Unless you're sub-selecting a column set, you don't need to specify the columns.

Upvotes: 2

chrisb
chrisb

Reputation: 52236

I'm not able to to reproduce the error (see below for a copy/paste-able example), can you make sure the the data you show is actually representative of your data?

In [137]: from StringIO import StringIO

In [138]: df = pd.read_csv(StringIO("""         x        y        ms
     ...: 0  20.5815  14.1821  333.3333
     ...: 1      NaN      NaN       350
     ...: 2  20.6112  14.2013  366.6667
     ...: 3      NaN      NaN  383.3333
     ...: 4  20.5349  14.2232       400"""), delim_whitespace=True)

In [140]: df = df.set_index(df.ms)

In [142]: df.interpolate(method='values')
Out[142]: 
                 x         y        ms
ms                                    
333.3333  20.58150  14.18210  333.3333
350.0000  20.59635  14.19170  350.0000
366.6667  20.61120  14.20130  366.6667
383.3333  20.57305  14.21225  383.3333
400.0000  20.53490  14.22320  400.0000

Upvotes: 0

Related Questions