CODEWITHSUNDEEP
pythonmongodbmongodb-querypymongo
Pierre
Pierre

Reputation: 6237

Provide a hint in bulk upserts

Is there a way to provide a hint for an upsert in a bulk in MongoDB / Python?

I would like to add a hint in a query like: Bulk.find(<query>).upsert().update(<update>).

I have tried:

Am I missing something?

Upvotes: 3

Views: 353

Answers (1)

Neil Lunn
Neil Lunn

Reputation: 151122

This is not so much about Bulk Operations but is rather about the general behavior of queries in "update" statements. See SERVER-1599.

So the same format of operations supported by the basic Op_Query which is linked to .find() has never been supported in update statements. This is also true of the Bulk API because the .find() method there is it's own method and belongs to the Bulk API where it is not related to the basic collection method, hence the lacking .hint() method.

So using the special forms as with $query does not work even with .update() in a basic form. But there is something you can do as of MongoDB 2.6 to influence the index chosen by the query.

The new addition here is "index filters", this allows you to set up a list of indexes to be considered for a given "query shape". The main definition here is through the planCacheSetFilter command. This allows you do do something like the following ( just in shell for brevity ):

db.junk.ensureIndex({ "b": 1, "a": 1 })

db.runCommand({
    "planCacheSetFilter": "junk",
    "query": { "a": 1 },
    "indexes": [
        { "b": 1, "a": 1 }
    ]
})

The values provided in the "query" argument there are irrelevant, but what is important is the "shape". So regardless of what data is being queried for, as long as the "shape" is basically the same then the filter set is considered. i.e:

db.junk.find({ "a": 1 }).explain(1).filterSet;     // returns true
db.junk.find({ "a": 2 }).explain(1).filterSet;     // returns true
db.junk.find({ "b": 1 }).explain(1).filterSet;     // returns false, different shape

Unlike the direct form of $hint, this will work with both .update() statements or in the Bulk .find().update() chain as a way to provide an index choice for the query operation.

Beware though that this is not a "permanent" setting, nor is it able to be isolated to a singular operation or sequence of operations. This "filter" will stay in the plan cache once set until the server instance is restarted. You can alternately clear it with the planCacheClearFilters command.

So until that JIRA Issue is resolved, "filters" are the only possible way like what you are asking to achieve without factoring in other queries to narrow down additional filtering parameters to optimize on the likely selected index.

Upvotes: 5

Related Questions