CODEWITHSUNDEEP
pythonjsonpandasnested
Charlie Guo
Charlie Guo

Reputation: 1

Convert pandas data frame to nested json

I have a Pandas Dataframe made from csv file, I want to convert it to a Custom Nested JSON

    df = pd.read_csv("/content/analog.csv", sep=",")

Dataframe is like:

datetime                key1    key2    key3        col1    col2    col3    ...  col100 
2020-08-27T02:28:00Z    WECP001 27001   con0000     1001    1002    1003    ...  1005   
2020-08-27T02:28:01Z    WECP001 27001   con0000     11      22      33      ...  99

I want to convert this df to a nested json like:

[ 
    {
       "datetime":"2020-08-27T02:28:00Z"
       "key1:"WECP001"
       "key2:"27001"
       "key3":"con0000"
       "DATA":{
               "col1":1001,
               "col2":1002,
               "col3":1003, 
               ...
               "col100":1005
              }
    },
    {
       "datetime":"2020-08-27T02:28:01Z"
       "key1:"WECP001"
       "key2:"27001"
       "key3":"con0000"
       "DATA":{
               "col1":11,
               "col2":22,
               "col3":33, 
               ...
               "col100":99
              }
    }
]

here is the code i have tired, but i get wrong result

cols = df.columns.difference(['datetime','key1','key2','key3'])
j = (df.groupby(['datetime','key1','key2','key3'])[cols]
       .apply(lambda x: x.to_dict('r'))
       .reset_index(name='DATA')
       .to_json(orient='records'))
print (j)

output is like:

{
   "datetime":"2020-08-27T02:28:00Z"
   "key1:"WECP001"
   "key2:"27001"
   "key3":"con0000"
   "DATA":[{
           "col1":1001,
           "col2":1002,
           "col3":1003, 
           ...
           "col100
           }]

the problem with my code is The "DATA" part became a List not a Dictionary thanks for help

Upvotes: 0

Views: 78

Answers (2)

Charlie Guo
Charlie Guo

Reputation: 1

import pandas as pd

df = pd.read_csv("***abc.csv", sep=",")

df2 = df.iloc[:, :4]
df2['DATA'] = df.iloc[:, 5:].to_dict(orient='records')

print(df2.to_json(orient='records', lines=True))

result:

{
   "datetime":"2020-08-27T02:28:00Z"
   "key1:"WECP001"
   "key2:"27001"
   "key3":"con0000"
   "DATA":{
           "col1":1001,
           "col2":1002,
           "col3":1003, 
           ...
           "col100": 
           }
}
{....}
{....}

Upvotes: 0

deadshot
deadshot

Reputation: 9061

Try this

df2 = df.iloc[:, :4]
df2['Data'] = df[[f'col{i}' for i in range(1, 4)]].to_dict(orient='records')
print(df2.to_json(orient='records'))

Output:

[
  {
    "datetime": "2020-08-27T02:28:00Z",
    "key1": "WECP001",
    "key2": 27001,
    "key3": "con0000",
    "Data": {
      "col1": 1001,
      "col2": 1002,
      "col3": 1003
    }
  },
  {
    "datetime": "2020-08-27T02:28:01Z",
    "key1": "WECP001",
    "key2": 27001,
    "key3": "con0000",
    "Data": {
      "col1": 11,
      "col2": 22,
      "col3": 33
    }
  }
]

Upvotes: 1

Related Questions