CODEWITHSUNDEEP
javamongodbupsert
Simon
Simon

Reputation: 19938

MongoDB: upsert with unique index

I must say I'm not really a mongoDB expert, but this is what I'm trying to do.

I have an app that would get a feed of "food" objects each time I switch on the app, this feed is received from an external web API service. I can only save the unique "food" ID for each of these objects due to ToS agreement with the external API service.

Now, each time I switch on the app, I would get the feed of food objects and I only want to save the food object ID ("foodId") once in my mongoDB. I have therefore decided to do an upsert each time I open the app. I'm not sure if this is the best way to write the upsert method as you can see the query params and the update params are the same and both relate to the "foodId":

                    Query query = new Query();
                    query.addCriteria(Criteria.where("foodId").in(foodIdfromApi));
                    Update update = new Update();
                    update.set("foodId", foodIdfromApi);
                    mongoTemplate.upsert(query, update, Food.class);

Since "foodId" is unique, I have created a unique index on "foodId"

This is from MongoDB: http://docs.mongodb.org/manual/reference/method/db.collection.update/#upsert-option

To prevent MongoDB from inserting the same document more than once, create a unique index on the name field. With a unique index, if multiple applications issue the same update with upsert: true, exactly one update() would successfully insert a new document.

The remaining operations would either:

update the newly inserted document, or fail when they attempted to insert a duplicate. If the operation fails because of a duplicate index key error, applications may retry the operation which will succeed as an update operation.

My app downloads the feed successfully the first time that it ran. But subsequently, it has been giving me the duplicate index key error each time I want to do an upsert (which is each time I open my app).

Write failed with error code 11000 and error message 'E11000 duplicate key error index

I do not want to remove the unique index as then there is a potential that duplicated food objects might exists. How do I get around this issue?

Upvotes: 4

Views: 7486

Answers (1)

Cuga
Cuga

Reputation: 17904

You're right in not removing the unique index. That's necessary in mongo, as you point out in the documentation.

To work around their choice to throw an error if there is an issue while attempting an upsert, you should catch the error and retry.

I'm not using mongo template, but here's an example using the Java driver. You can try something similar:

MongoCollection<Document> collection = mongo.getCollection("Revenue");
    UpdateResult result;
    try {
        result = collection.replaceOne(searchDoc, repsertDoc, new UpdateOptions().upsert(true));
    } catch (MongoException e) {
        // Retry once on error, can sleep here too if you'd like
        result = collection.replaceOne(searchDoc, repsertDoc, new UpdateOptions().upsert(true));
        // If this attempt fails, can always re-try again or fail
    }

Upvotes: -1

Related Questions