CODEWITHSUNDEEP
pandasdataframelinear-regressionpredictionstatsmodels
orome
orome

Reputation: 48466

Adding statsmodels 'predict' results to a Pandas dataframe

It is common to want to append the results of predictions to the dataset used to make the predictions, but the statsmodels predict function returns (non-indexed) results of a potentially different length than the dataset on which predictions are based.

For example, if the test dataset, test, contains any null entries, then

mod_fit = sm.Logit.from_formula('Y ~ A B C', train).fit()
press = mod_fit.predict(test)

will produce an array that is shorter than the length of test, and cannot be usefully appended with

test['preds'] = preds

And since the result of predict is not indexed, there is no way to recover the rows to which the results should be attached.

What is the idiom for associating predict results to the rows from which they were generated? Is there, perhaps, a way to get predict to return a dataframe that preserves the indices of its argument?

Upvotes: 2

Views: 3531

Answers (1)

jseabold
jseabold

Reputation: 8283

Predict shouldn't drop any rows. Can you post a minimal working example where this happens? Preserving the pandas index is on my radar and should be fixed in master soon.

https://github.com/statsmodels/statsmodels/issues/1501

Edit: Nevermind. This is a known issue. https://github.com/statsmodels/statsmodels/issues/1352

Upvotes: 2

Related Questions