CODEWITHSUNDEEP
phpmongodbdoctrinedatabasenosql
mTuran
mTuran

Reputation: 1834

Optimizing MongoDB Query on Doctrine

This query takes very much time in my application. Database contains about 2 million records.

    $results = $queryBuilder
        ->field('extra.targetPlayerId')->exists(FALSE)
        ->field('permission')->equals('public')
        ->field('time')->gt($weekEndTime) // variable is timestamp
        ->field('time')->lte($startTime) // variable is timestamp
        ->map(
            'function() {
                var total = 0;
                if (this.comments) {
                    total += this.comments.length;
                }
                if (this.likes) {
                    total += this.likes.length;
                }                    
                if (total > 0) {
                    emit(this.playerId, total);
                }
            }'
        )
        ->reduce(
            'function(key, values) {
                var total = 0;
                for (value in values) {
                    total += values[value];
                };
                return total;
            }'
        )->getQuery()->execute();

How can i optimize this query ? Can you give me suggestion for indexes ?

Upvotes: 1

Views: 1024

Answers (1)

AD7six
AD7six

Reputation: 66268

->field('extra.targetPlayerId')->exists(FALSE)

That right there is probably your biggest obstacle to making the query faster.

from the docs:

Before v2.0, $exists is not able to use an index. Indexes on other fields are still used. $exists is not very efficient even with an index, and esp. with {$exists:true} since it will effectively have to scan all indexed values.

Perhaps consider adding a boolean field such that you can test for true/false instead of exists - and include that field in the index.

Or even move that particular condition into your map function:

function() {
    var total = 0;

    if (!this.extra || !this.extra.targetPlayerId) {
        return; //bail - count nothing
    }

    if (this.comments) {
        total += this.comments.length;
    }
    if (this.likes) {
        total += this.likes.length;
    }                    
    if (total > 0) {
        emit(this.playerId, total);
    }
}

Indexing isn't a black and white answer, for example, if you have a lot of data:

{
    time: 1,
    permission: 1,
    extra.hasTargetPlayer: 1
}

would probably work quite well - since it would permit mongo to focus on the right date range as a first criteria.

Aside from that it may be more appropriate to simply change your data schema. Are you storing the result of that query?

i.e. query once and store the result, so that for any subsequent requests you can just do:

db.appstats.findOne({_id: "201152"})

where _id is a year and weeknumber. If you are generating stats by week, every week - you can do the heavy query on a schedule (cron) and therefore users aren't affected by the speed of the query to actually calculate the results.

Upvotes: 4

Related Questions