CODEWITHSUNDEEP
pythonjulia
tuckdales
tuckdales

Reputation: 43

Import nested Python dict in Julia

I have developed a Python script which loads data from excel files (.xls/m, .csv etc) processes it (stores it in nested dictionary) and uses that data to solve a non convex optimization problem.

After discovering the amazing NonConvex.jl package, I would like to take advantage of the algorithms and obviously the speed up from Julia.

Since the data loading module is already written in Python, I would like to continue using that. For non convex optimization i would like Julia to work with the pre processed data (from Python script and in the form of nested Python dictionary).

Is there a recommended way to export the nested dictionary from Python and import into Julia.

Thanks!

I have used Pickle to export nested dictionaries from Python and import the pickle file in Julia using Pickle.jl, and didn’t succeed.

I have also tried exporting Python dictionary as a json object but failed as my nested dictionary contains Int64 data type and json.dump doesn’t work with it.

Upvotes: 2

Views: 337

Answers (2)

Shayan
Shayan

Reputation: 6295

I went through this procedure successfully:

  1. Python:

    >>> dictionary = {
...     1:{"key1":[1, 2, 3]},
...     2:[3, 5]
... }

>>> import pandas as pd
>>> pd.to_pickle(dictionary, "thepickle.pickle")

  2. Then, using the Pandas.jl in Julia:

    julia> using Pandas: read_pickle

julia> dictionary = read_pickle("thepickle.pickle")
Dict{Any, Any} with 2 entries:
  2 => [3, 5]
  1 => Dict{Any, Any}("key1"=>[1, 2, 3])

Note that you can use either pickle or pkl as the file type suffix.

Upvotes: 1

Przemyslaw Szufel
Przemyslaw Szufel

Reputation: 42214

You can load pickle via PyCall.

I just managed to run the following code:

import numpy, pickle
d = { "A":[1,2,3], "B":np.array([4,5,6], dtype='int64') }
with open("c:\\temp\\file.bin","wb")  as f :
   pickle.dump(d,f)

Now I try to load in Julia via PyCall and works like a charm:

julia> using PyCall

julia> pickle = pyimport("pickle");

julia> open("c:\\temp\\file.bin") do f
       d = pickle.load(f)
       end
Dict{Any, Any} with 2 entries:
  "B" => [4, 5, 6]
  "A" => [1, 2, 3]

Upvotes: 2

Related Questions