CODEWITHSUNDEEP
pythonpandasdataframepickle
s.paszko
s.paszko

Reputation: 732

Is it safe to save Dataframe wrapped with other object as pickle?

I would like to save Dataframe as pickle together with some other data. I could use a dict wrapper for it or specialized class. Is it safe to to save/load these DataFrame using module pickle dump/load functions or should I always save Dataframe with pandas to_pickle/read_pickle methods? What do i lose, when i save/load without pandas code?

Example what i would like to do (using dict to wrap all data) :

import pandas as pd
import pickle
df = pd.DataFrame([[1, 2], [3, 4]], columns=list('AB'), index=['x', 'y'])
dictDf = {'Important data' : False, 'Other information' : 'Important', 'Df' : df}

# Save
with open('dictDf.pickle','wb') as f:
    pickle.dump(dictDf, f)

with open('dictDf.pickle','rb') as f:
    dictDfReaded = pickle.load(f)

Upvotes: 0

Views: 521

Answers (1)

s.paszko
s.paszko

Reputation: 732

In other question (What's the diffrence between pandas pd.to_pickle and pickle module pickle.dump, when saving DataFrame?) there is answer(https://stackoverflow.com/a/74458675/2516697) about diffrences between module pandas pickle functions and module pickle functions.

So generally speaking, except from some specific cases(python2.7, protocol==5 and bz2/xz compresssion), it's safe to save/load DataFrame packed inside other object as pickle with module pickle functions.

Upvotes: 1

Related Questions