CODEWITHSUNDEEP
pythonmongodbpymongoupsert
Sushant Gupta
Sushant Gupta

Reputation: 9458

pymongo- upsert not able to perform insertion with $set operation

I am having an empty collection and have thousands of entries to process (entries might have redudancy for which I want to use both updates and inserts). The python code (using pymongo) I wrote:

for mydoc in alldocs:
   key = {'myid': mydoc['myid']}
   data = process_doc(mydoc)    # returns simple dictionary
   db.mydocs.update(key, {"$set": data}, upsert = True)

The following code is unable to perform any insert operations. The collection still remains empty. But when I remove $set and use simply data, it works fine. Can't I use $set in upsert? The reason why I want $set was so that pre-existing fields for a BSON doesn't get affected. Can someone please guide. I really can't figure out what to do.

Reproducable code:

from pymongo import Connection
DB_CONTENT_BASE_KEY = 'contentbase'

def connect_to_db(dbname, hostname = 'localhost', portno = 27017, **kwargs):
    connection = Connection(hostname, portno)
    dbConnection = connection[dbname]
    return dbConnection

class MetawebCustomCollectionBuilder(object):
    # key ought to be a dictionary to filter results from contentbase.
    def __init__(self, inDbConfig, outDbConfig, key = {}, verbose = False):
        self.verbose = verbose
        self.inDbConfig = inDbConfig
        self.inDb = connect_to_db(**inDbConfig)
        self.outDbConfig = outDbConfig
        self.outDb = connect_to_db(**outDbConfig)
        self.inDbContentBase = self.inDb[self.inDbConfig[DB_CONTENT_BASE_KEY]]
        self.outDbContentBase = self.outDb[self.outDbConfig[DB_CONTENT_BASE_KEY]]
        self.key = key
        self.in_db_collection_constraints()
        self.out_db_collection_constraints()

    def in_db_collection_constraints(self):
        self.inDbContentBase.ensure_index('mid')
        if self.verbose: print("Assured index on mid for inDbContentBase...")

    def out_db_collection_constraints(self):
        self.outDbContentBase.ensure_index('mid')
        if self.verbose: print("Assured index on mid for outDbContentBase...")

    def process_in_record(self, inRecord):
        outRecord = inRecord # [YET TO] continue from here...
        return outRecord

    def transit_collection(self):
        for record in self.inDbContentBase.find(self.key):
            outRecord = self.process_in_record(record)
            key = {'mid':outRecord['mid']}
            data = outRecord
            print key
            self.outDbContentBase.update(key, {"$set": data}, True)
        if self.verbose: print 'Done with transiting collection from in DB to out DB'

    def cleanup_out_collection(self):
        pass

    def in_db_sandbox(self):
        # To have tests and analytics placed in here corresponding to inDb.
        pass

if __name__ == '__main__':
    inDbConfig = {'dbname':'metaweb', 'contentbase': 'content'}
    outDbConfig = {'dbname': 'similarkind', 'contentbase': 'content'}
    mccb = MetawebCustomCollectionBuilder(inDbConfig, outDbConfig, verbose = True)
    mccb.transit_collection()

There must be a prexisting database inDb. From this collection I want to create a new modified collection.

Upvotes: 3

Views: 6142

Answers (1)

user2665694
user2665694

Reputation:

Your claim is wrong

>>> import pymongo
>>> c = pymongo.Connection()

>>> db = c.mydb
>>> db.mydocs.find().count()
0
>>> db.mydocs.update({'myid': '438'}, {"$set": {'keyA':'valueA'}}, upsert = True)
>>> db.mydocs.find().count()
1
>>> db.mydocs.find_one()
{u'myid': u'438', u'keyA': u'valueA', u'_id': ObjectId('504c2fd1a694cc9624bbd6a2')}

Upvotes: 8

Related Questions