CODEWITHSUNDEEP
pythonpandaspickle
edenPan
edenPan

Reputation: 81

How to load a pickle by pandas not from the file

since previously the code:

def read_pickle_packet(self, sf):
    body = self._read_payload(sf)

    return pickle.load(body)

has an error:ImportError: No module named indexes.base. which was caused by the pandas version inconsistent between server and client. I solve the problem by changing the code like this:

def read_pickle_packet(self, sf):
    body = self._read_payload(sf)
    filehandler = open("tempFile.pkl", 'w')
    filehandler.write(body)
    filehandler.close()
    return pd.read_pickle("tempFile.pkl")

It can solve the problem. But my question is there any more quick way to load the pickle object rather than store it into filesystem and then read it. It cost time and memory.But i could not found some clue by google "pandas read pickle from object".
Thank you for any suggestion.

Upvotes: 3

Views: 2366

Answers (1)

Kota Mori
Kota Mori

Reputation: 6740

It seems pd.read_pickle is only for reading from a file. If you don't need to use the pandas library, you can skip saving to disk if you use pickle.dumps and pickle.loads (https://docs.python.org/3/library/pickle.html). I am not sure if this works in your case since I am not clear about your _read_payload function, though.

import pandas as pd
import pickle

x = pd.DataFrame({"x": [1,2,3], "y": [4,5,6]})
print(x)

obj = pickle.dumps(x)
z = pickle.loads(obj)
print(z)

Upvotes: 1

Related Questions