CODEWITHSUNDEEP
mongodbperformanceinsert
hudao
hudao

Reputation: 21

mongodb insert really slow

i use mongodb to manage device log datas. Right now, it has over one million documents. the document contains more than 30 fields which combine with embed fields. Now, it's really slow when i insert new documents. The insert cost more than 1000ms. From the slow query ops, i get the logs like this:

{
    "op" : "insert",
    "ns" : "xxx.LogDeviceReport",
    "query" : {
        "_id" : ObjectId("xxxx"),
        "deviceId" : ObjectId("xxxx"),
        "en" : "xxxxxx",
        "status" : 1,
        'other fields, more than 30 fields...'
        ...
        ...

    },
    "ninserted" : 1,
    "keyUpdates" : 0,
    "writeConflicts" : 0,
    "numYield" : 0,
    "locks" : {
        "Global" : {
            "acquireCount" : {
                "w" : NumberLong(2)
            }
        },
        "MMAPV1Journal" : {
            "acquireCount" : {
                "w" : NumberLong(3)
            }
        },
        "Database" : {
            "acquireCount" : {
                "w" : NumberLong(2)
            }
        },
        "Collection" : {
            "acquireCount" : {
                "W" : NumberLong(1)
            },
            "acquireWaitCount" : {
                "W" : NumberLong(1)
            },
            "timeAcquiringMicros" : {
                "W" : NumberLong(1477481)
            }
        },
        "oplog" : {
            "acquireCount" : {
                "w" : NumberLong(1)
            }
        }
    },
    "millis" : 977,
    "execStats" : {

    },
    "ts" : ISODate("2016-08-02T22:01:01.270Z"),
    "client" : "xxx.xxx.xxxx",
    "allUsers" : [
        {
            "user" : "xxx",
            "db" : "xxx"
        }
    ],
    "user" : "xx@xx"
}

I checked the index, like this:

[
    {
        "v" : 1,
        "key" : {
            "_id" : 1
        },
        "name" : "_id_",
        "ns" : "xxx.LogDeviceReport"
    },
    {
        "v" : 1,
        "key" : {
            "time" : 1
        },
        "name" : "time_1",
        "ns" : "xxx.LogDeviceReport",
        "expireAfterSeconds" : 604800,
        "background" : true
    }
]

Only an _id index and a ttl index by time, no any other indexes.

I guess the 'query' slow the operate. In mongodb doc, it tells that only the _id will be checked the unique, but in the logs, all fields in the 'query', does it matter?

if not this reason, what makes it so slow? Can any one help me ?

Upvotes: 2

Views: 15225

Answers (1)

rajadilipkolli
rajadilipkolli

Reputation: 3601

If you are using mongodb 3+ you can consider using WiredTiger as storage engine than MMAPV1 which is being used in your case.

I have personally saw a 4x improvement when I have inserted up to 156000 documents in a single go.

MMAPV1 took around 40 min and when I switched to WiredTiger same task was completed in 10 min.

Please check this link from MongoDB blog for more information

Note :: This is only from MongoDB 3.0 +

Upvotes: 3

Related Questions