Selecting Column from pandas Series

Question

I have a Series named 'graph' in pandas that looks like this:

Wavelength
450      37
455      31
460       0
465       0
470       0
475       0
480     418
485    1103
490    1236
495     894
500     530
505      85
510       0
515     168
520       0
525       0
530     691
535     842
540    5263
545    4738
550    6237
555    1712
560     767
565     620
570       0
575     757
580    1324
585    1792
590     659
595    1001
600     601
605     823
610       0
615     134
620    3512
625     266
630     155
635     743
640     648
645       0
650     583
Name: A1, dtype: object

I am graphing the curve using graph.plot(), which looks like this :

The goal is to smooth the curve. I was trying to use the Savgol_Filter, but to do that I need to separate my series into x & y columns. As of right now, I can acess the "Wavelength" column by using graph.index, but I can't grab the next column to assign it as y.

I've tried using iloc and loc and haven't had any luck yet.

Any tips or new directions to try?

Answer 1

You don't need to pass an x and a y to savgol_filter. You just need the y values which get passed automatically when you pass graph to it. What you are missing is the window size parameter and the polygon order parameter that define the smoothing.

from scipy.signal import savgol_filter
import pandas as pd

# I passed `graph` but I could've passed `graph.values`
# It is `graph.values` that will get used in the filtering
pd.Series(savgol_filter(graph, 7, 3), graph.index).plot()

---

To address some other points of misunderstanding

1. graph is a pandas.Series and NOT a pandas.DataFrame. A pandas.DataFrame can be thought of as a pandas.Series of pandas.Series.
2. So you access the index of the series with graph.index and the values with graph.values.

3. You could have also done

   import matplotlib.pyplot as plt
   
   plt.plot(graph.index, savgol_filter(graph.values, 7, 3))
   

Answer 2

As you are using Series instead of DataFrame, some libraries could not access index to use it as a column.
Use:

df = df.reset_index()

it will convert the index to an extra column you can use in savgol filter or any other.