Reputation: 9893
I want to upsert(update or insert) a list of record
, in fact I know that mongodb support bulk
operation since mongodb3.0.
I want to know whether mongoengine
support bulk upsert operation
in mongoengine (0.10.0)
.
if not, I want to know how to upsert
a list of record, I know mongoengine
support insert
batch like this:
class User(Document):
username = StringField(required=True)
password = StringFiedl(required=True)
meta = {'db_alias': 'user_info', 'collection': 'user',
'indexes': [{'fields': ['username'], 'unique': True}]
}
def save_users(self, users):
Users.objects.insert(users) # raise mongoengine.errors.NotUniqueError
Upvotes: 3
Views: 3561
Reputation: 59604
You can use mongo.collection.Collection.bulk_write
:
operations = [
pymongo.ReplaceOne({'username': user. username}, user.to_mongo(), upsert=True)
for user in users
]
result = User._get_collection().bulk_write(operations)
Upvotes: 2
Reputation: 50406
You can actually use the bulk operations API directly by accessing the underlying collection object from the pymongo driver MongoEngine uses. MongoDB itself has supported bulk operations since version 2.6. There are newer methods to access these since v3 of the pymongo driver, but the basic methods have been around since the corresponding driver update to the 2.6 Server version ( pymongo 2.7 ).
To get this with MongoEngine you can call the undocumented ._get_collection()
from your class to return the collection object:
bulk = Users._get_collection().initialize_ordered_bulk_op()
for user in users: # where users is a list of dicts containing data to work on
bulk.find({ "matchField": user['matchField'] }).upsert().replace_one(user)
bulk.execute()
Or any other usage of bulk methods such as .update_one()
which you probably want. And .upsert()
is a chained method that modifies such update statements.
You are working with raw python objects here as there is no direct equivalent in MongoEngine itself. But you can use the operations by accessing the methods from the underlying driver
Upvotes: 7