CODEWITHSUNDEEP
pythonfiledictionarysavepickle
mathguy
mathguy

Reputation: 1518

Saving and loading a complicated python dictionary without resorting to csv files

I have been dealing with simple dictionaries for a while and the only way I know that can save/load them is via to_csv() and pd.read_csv() in pandas. Because csv is the only way I know that can get the job done, sometimes I may encounter dictionaries whose key may be associated with complicated stuff that cannot be saved/loaded properly in csv, which is why I am asking for help here.

Here is an example of a slightly complicated dictionary that I just came up with:

import numpy as np
import pandas as pd

simple_dict1 = {'dict1': {'item1': None,
                          'item2': False,
                          'item3': 'hello world',
                          'item4': 42.42,
                          'item5': [f'item {i}' for i in range(42)],
                          'item6': {f'itme {i}': i for i in range(42)},
                          'item7': np.random.rand(3,42),
                          'item8': pd.DataFrame(np.random.rand(3,42), columns = [f'col {i+1}' for i in range(42)]) 
                          },
                'dict2': {'item1': True,
                          'item2': 'hello',
                          'item3': 24.24,
                          'item4': [f'item {i}' for i in range(24)],
                          'item5': {f'itme {i}': i for i in range(24)},
                          'item6': np.random.rand(2,24),
                          'item7': pd.DataFrame(np.random.rand(2,24), columns = [f'col {i+1}' for i in range(24)]) 
                          },
                'dict3': {'item1': (10,20),
                          'item2': [],
                          'item3': {},
                          'item4': (),
                          'item5': {i for i in range(4)},
                          'item6': {f'itme {i}': {} for i in range(24)},
                          },
                'list1': [type(item) for item in ({}, [], ())]
                }

My goal is create two functions:

function 1: takes a relative/absolute path and a dictionary as inputs, and save the said dictionary to the said path.

function 2: takes a relative/absolute path as an input, read the dictionary from the said path, and return the said dictionary. This dictionary needs to be identical to the original dictionary that was saved.

This question may be too much to ask, but I really have no in depth knowledge how to save/load complicated dictionary like the one above. Thank you in advance.

EDIT: changed one of the tag to pickle, since pickle may be able to solve the problem.

Upvotes: 0

Views: 41

Answers (1)

Matthew Schinckel
Matthew Schinckel

Reputation: 35619

You can use the pickle standard library to load and save arbitrary data structures.

https://docs.python.org/3/library/pickle.html

There are caveats: as shown on that page, do not unpickle untrusted data, and the pickle format may change between different python versions.

Upvotes: 1

Related Questions