CODEWITHSUNDEEP
pythonpandasstatsmodels
bikuser
bikuser

Reputation: 2103

working with missing data in pandas python

I have a time series data frame with x and y as columns name. In my dataframe y is incomplete series and x is complete series. I tried to fit linear regression model between x and y. My intetion is to fill data gap by usig this model. my sample codes are:

import statsmodels.formula.api as sm
result = sm.ols(formula="y ~ x", data=df_rg).fit()
df_rg['y'][df_rg['y'].fillna(predict(df_rg['x'])

where df_rg is my dataframe with datetime as an index.

my dataframe looks like:

date         x          y
1957-07-31  18.845161   NaN
1957-08-31  18.080645   NaN
1957-09-30  16.156667   NaN
1957-10-31  12.324194   NaN
1957-11-30  8.948333    NaN
1957-12-31  6.253226    NaN
.............................
.............................
2015-03-31  8.316129    20.088710
2015-04-30  10.408333   22.203333
2015-05-31  14.832258   25.258065
2015-06-30  16.815000   26.453333
2015-07-31  18.141935   26.835484
2015-08-31  18.450000   26.637097
2015-09-30  17.016667   26.513333
2015-10-31  11.898387   22.906452
2015-11-30  9.133333    19.376667
2015-12-31  5.366129    14.441935

Upvotes: 2

Views: 488

Answers (1)

Graipher
Graipher

Reputation: 7206

Maybe use the dataframe without the NA values for the fitting, using pandas.DataFrame.dropna?

import statsmodels.formula.api as sm
result = sm.ols(formula="y ~ x", data=df_rg.dropna()).fit()
df_rg['y'] = df_rg['y'].fillna(predict(df_rg['x']))

Upvotes: 3

Related Questions