CODEWITHSUNDEEP
pythonjson
Ank
Ank

Reputation: 6270

Skipping broken jsons python

I am reading JSON from the database and parsing it using python. 

cur1.execute("Select JSON from t1")
dataJSON = cur1.fetchall()
for row in dataJSON:
 jsonparse = json.loads(row)

The problem is some JSON's that I'm reading is broken. I would like my program to skip the json if its not a valid json and if it is then go ahead and parse it. Right now my program crashes once it encounters a broken json. T1 has several JSON's that I'm reading one by one.

Upvotes: 2

Views: 12841

Answers (3)

ohsoifelse
ohsoifelse

Reputation: 683

Try this:

def f(x):
  try:
    return json.loads(x)
  except:
    pass

json_df = pd.DataFrame()
json_df = df.join(df["error"].apply(lambda x: f(x)).apply(pd.Series))

After JSON loads, I also wanted to convert each key-value pair from JSON to a new column (all JSON keys), so I used apply(pd.Series) in conjunction. You should try this by removing that if your goal is only to convert each row from a data frame column to JSON.

Upvotes: 0

Jon Clements
Jon Clements

Reputation: 142156

Update

You're getting an expecting string or buffer - you need to be using row[0] as the results will be 1-tuples... and you wish to take the first and only column.

If you did want to check for bad json

You can put a try/except around it:

for row in dataJSON:
    try:
        jsonparse = json.loads(row)
    except Exception as e:
        pass

Now - instead of using Exception as above - use the type of exception that's occuring at the moment so that you don't capture non-json loading related errors... (It's probably ValueError)

Upvotes: 6

SheetJS
SheetJS

Reputation: 22905

If you just want to silently ignore errors, you can wrap json.loads in a try..except block:

try: jsonparse = json.loads(row)
except: pass

Upvotes: 1

Related Questions