CODEWITHSUNDEEP
pythonpython-3.xmongodb
Justin
Justin

Reputation: 331

How can I avoid errors inserting JSON data into MongoDB

I am downloading JSON data and trying to save it into MongoDB (local server instance which is working when inputting simple JSON). When I try with more complex JSON data I am getting the following error:

TypeError: document must be an instance of dict, bson.son.SON,bson.raw_bson.RawBSONDocument, or a type that inherits from collections.MutableMapping

If I print out the post and insert it into MongoDB via the Robo 3T it is imported without any errors.

Here is my code:

    from pymongo import MongoClient
    import datetime
    import requests

    #Connect to mongo
    client = MongoClient('192.168.1.120', 27017)
    db = client.test_database
    collection = db.test_collection

    #get JSON data
    ticker = '0511.HK'
    url = 'https://query2.finance.yahoo.com/v10/finance/quoteSummary/' + \
  ticker + '?formatted=true&crumb=cmEFpzsN8.l&lang=en-CA&region=CA&modules=summaryProfile&corsDomain=ca.finance.yahoo.com'
    r = requests.get(url)
    resp = str(r.text)
    post = resp[1:-1]

    posts = db.posts
    post_id = posts.insert_one(post).inserted_id
    print(post_id)

Upvotes: 0

Views: 98

Answers (1)

Dave
Dave

Reputation: 11899

Your error message is telling you what to do. You need to provide a dictionary or other mapping object. Instead, you're sending a string:

resp = str(r.text)
post = resp[1:-1]

I'm not 100% sure why you're doing that second line, but whatever.

Instead of brute-forcing the response data into a string, you want to parse it into a dictionary. requests has a built in method for doing that:

post = r.json()

This will result in post being a dictionary, and should work when sent to MongoDB.

Upvotes: 1

Related Questions