CODEWITHSUNDEEP
mongodbamazon-web-servicesamazon-ec2
Richard Mao
Richard Mao

Reputation: 302

Mongodb 3.2.6 pauses a lot when heavy writes

I have one collection with 0.3 billion of document(212GB data) with no index ( _id index is 11GB ) . 32GB RAM , 8 CPU Amazon Linux AMI m4.2xlarge 500GB SSD EBS volume. 3-4Mb writes per second , Mongodb 3.2.6 with wiredTiger storage engine , block_compressor is snappy . I have optimized according to the instructions from Mongodb documentation of installation on Amazon EC2 .

What I found is , server inconsistently use lots of CPU to 700% or 30% of total CPU . All writes and reads was paused for around 5-20s . I read lots of material online and guess it might be the reasons of wiredTiger snapshot checkpoints process or data compression process.

The 5-20s pause is really annoying and other applications will be also impacted .

My question is what are the alternatives to resolve this issue ? Will it become even worst when my data is growing up to 1 billion documents (no index needed at this moment , just heavy writes )?

Possible solutions I can come up with

  1. Shading (my most jobs are heavy writes , no reads , no query , no index needed at this moment )

  2. revert back to MMAPv1

  3. more tuning or optimization

  4. using another database

How did I resolve this issue ?

I changed amazon ebs volume from ext4 to xfs format by following this How to create and mount an XFS file system on Linux and Making an Amazon EBS Volume Available for Use.

Now the system looks like this . No pause at all . screenshot from mongostat screenshot from iotop

Upvotes: 1

Views: 286

Answers (1)

Kiril
Kiril

Reputation: 998

This sounds an awful lot like an ext4 problem. When running WiredTiger you should use XFS.

Upvotes: 3

Related Questions