CODEWITHSUNDEEP
pythonpandaspicklenetflix-metaflow
crypdick
crypdick

Reputation: 19786

ModuleNotFoundError: No module named 'pandas.core.indexes.numeric' using Metaflow

I used Metaflow to load a Dataframe. It was successfully unpickled from the artifact store, but when I try to view its index using df.index, I get an error that says ModuleNotFoundError: No module named 'pandas.core.indexes.numeric'. Why?

I've looked at other answers with similar error messages here and here, which say that this is caused by trying to unpickle a dataframe with older versions of Pandas. However, my error is slightly different, and it is not fixed by upgrading Pandas (pip install pandas -U).

Upvotes: 39

Views: 51713

Answers (5)

Manish Arya
Manish Arya

Reputation: 1

try this if you using pandas > 2.0.0

Import necessary libraries:

import os
import pandas as pd

Get the absolute path to the current directory:

current_directory = os.path.abspath(os.getcwd())

Construct the absolute path to the pickle file ('file.pkl' in this case):

file_path_file = os.path.join(current_directory, 'file.pkl')

Use pandas to read the pickle file directly:

file_df = pd.read_pickle(file_path_file)

Upvotes: 0

Clinton Woods
Clinton Woods

Reputation: 249

So i dont know why this works but joblib.load was failing to read the pickle with the same error "module named 'pandas.core.indexes.numeric'" then i installed prefect and simple_salesforce and some how it now works... not sure why but i think worth mentioning

Upvotes: 0

MR42
MR42

Reputation: 51

Try using pd.compat.pickle_compat.load() as that was only solution in my case:

import pandas as pd

df = pd.compat.pickle_compat.load('file.pkl')

Upvotes: 5

Mike T
Mike T

Reputation: 43612

Try using the pandas.read_pickle() method to load the file instead of the pickle module:

import pandas as pd

df = pd.read_pickle("file.pkl")

The pandas method should provide compatibility to read older files, and "is only guaranteed to be backwards compatible to pandas 0.20.3 provided the object was serialized with to_pickle." My tests with pandas-1.x show it can also read some files written from the pickle module too.

Upvotes: 46

crypdick
crypdick

Reputation: 19786

This issue is caused by the new Pandas 2.0.0 release breaking backwards compatibility with Pandas 1.x, although I don't see this documented in the release notes. The solution is to downgrade pandas to the 1.x series: pip install "pandas<2.0.0"

Upvotes: 46

Related Questions