CODEWITHSUNDEEP
pythonpandasderivative
Ilya
Ilya

Reputation: 541

Compute derivative in pandas?

I'm trying to compute a derivative of a pandas Series, i.e. (x[n] - x[n-1]) / (t[n] - t[n-1]).

I'm sorry if this question is a duplicate, and I know there are many similar-sounding questions already out there, but I see that most people who ask for a derivative actually mean a finite difference. To recap: a finite difference is x[n] - x[n-1] (not what I want). Note also that pandas.Series.diff computes a finite difference, and the documentation does not claim to compute a derivative.

Here is an example:

>>> s = pd.Series([1, 2, 3, 4], [1, 2, 4, 7])
>>> s.diff() / np.gradient(s.index)
1         NaN
2    0.666667
4    0.400000
7    0.333333
dtype: float64

Is there no single function call that does this? I don't want to resample, either -- that's overkill.

Note that:

Upvotes: 0

Views: 2087

Answers (2)

Nick ODell
Nick ODell

Reputation: 25409

I'm trying to compute a derivative of a pandas Series, i.e. (x[n] - x[n-1]) / (t[n] - t[n-1]).

This is the finite difference of the column divided by the finite difference of the index.

e.g.

>>> s = pd.Series([1, 2, 3, 4], [1, 2, 4, 7])
>>> s.diff() / s.index.to_series().diff()
1         NaN
2    1.000000
4    0.500000
7    0.333333
dtype: float64

Note that this will give you slightly different answers than the solution using np.gradient(). np.gradient() uses a two-sided approximation for the derivative. Documentation

Here's an example showing that np.gradient() isn't just t[n] - t[n-1]. It takes both the previous and next element into account when finding the approximation to the derivative.

>>> np.gradient([1, 2, 1000])
array([  1. , 499.5, 998. ])

Upvotes: 1

Ouyang Ze
Ouyang Ze

Reputation: 490

Usually the best way to do this kind of very specific operation is to leverage fundamentals of pandas and just write it yourself:

(
 pd.DataFrame({'x':[1, 2, 3, 4], 't':[1, 2, 4, 7]})
 .assign(xnm1=lambda d: d.x.shift(-1), 
         tnm1=lambda d: d.t.shift(-1))
 .pipe(lambda d: (d.x - d.xnm1) / (d.t - d.tnm1))
 )

Upvotes: 0

Related Questions