CODEWITHSUNDEEP
pythonjsonpython-3.xpostgresqllist
Demi Dimitrova
Demi Dimitrova

Reputation: 369

How to iterate trough JSON list in python and insert into PostgreSQL?

I am a bit stuck and the solutions I can think of will end in spaghetti code.

I am having the following JSON, I create an object of it in python 3 and I have to insert the values into PostgreSQL.

{
    "name": "test",
    "country": "DE",
    "time": "21-Oct-2019 (09:58:01.694763)",
    "toCurrency": ["EUR", "USD", "GBP"],
    "fromCurrency": ["DKK", "DKK", "DKK"],
    "buyMargin": ["1.2800", "1.2800", "2.0000"],
    "sellMargin": ["1.2800", "1.2800", "2.0000"],
    "unit": "M100" 
}

I am getting the exception: can only concatenate str (not "list") to str

At the same time the other values of 'name', 'country', 'time' should be inserted every time.

Currently I can insert this JSON: 

{
"name": "test",
"country": "DE",
"time": "21-Oct-2019 (09:58:01.694763)",
"toCurrency": "EUR",
"fromCurrency": "DKK",
"buyMargin": "1.2800",
"sellMargin": "1.2800",
"unit": "M100" 
}

Upvotes: 0

Views: 195

Answers (2)

Trollsors
Trollsors

Reputation: 492

Convert your data to a dataframe using pandas. Then using sqlalchemy, store this dataframe as a table in PostGre.

import pandas as pd
from sqlalchemy import create_engine
d1 = {
    "name": "test",
    "country": "DE",
    "time": "21-Oct-2019 (09:58:01.694763)",
    "toCurrency": ["EUR", "USD", "GBP"],
    "fromCurrency": ["DKK", "DKK", "DKK"],
    "buyMargin": ["1.2800", "1.2800", "2.0000"],
    "sellMargin": ["1.2800", "1.2800", "2.0000"],
    "unit": "M100" 
}
df1 = pd.DataFrame(d1)
engine = create_engine(
        'postgresql://username:password@host:port/database')
df1.to_sql(tablename, engine, if_exists='append', index=False)

Your Dataframe will look as follows. And will similarly be stored as a table in PostGre

enter image description here

Upvotes: 1

Kostas Charitidis
Kostas Charitidis

Reputation: 3113

If I understand correct and your toCurrency, fromCurrency, buyMargin and sellMargin have equal number of elements which they should, the following should work gor you:

j = {
    "name": "test",
    "country": "DE",
    "time": "21-Oct-2019 (09:58:01.694763)",
    "toCurrency": ["EUR", "USD", "GBP"],
    "fromCurrency": ["DKK", "DKK", "DKK"],
    "buyMargin": ["1.2800", "1.2800", "2.0000"],
    "sellMargin": ["1.2800", "1.2800", "2.0000"],
    "unit": "M100"
}

for idx, val in enumerate(j['toCurrency']):
    print(j['toCurrency'][idx], j['fromCurrency'][idx], j['buyMargin'][idx], j['sellMargin'][idx])

So your insert statement should be in the position of the print and your columns should be accordingly.

Upvotes: 1

Related Questions