CODEWITHSUNDEEP
node.jscouchdbcouchdb-nano
Lukasz Kujawa
Lukasz Kujawa

Reputation: 3096

Very slow inserts with CouchDB?

I wanted to check how many more inserts CouchDB can handle when compared to MySQL. My test was simple: for 10 seconds keep inserting {firstName: "Testing 001", lastName: "Testing 002"} and compare the number of documents/rows. The results I got were far from my expectations:

Please correct me if I'm wrong but should a NoSQL always outperform a relational database in simple operations? I would not expect such a dramatic difference. Perhaps my test was naive and I shouldn't be comparing those databases in such a way? I know that MySQL driver has an access to the connection pool and doesn't have to re-create a TCP connection on every request but does it render such a big difference?

Should CouchDB insert be so slow and if not how to do it right?

I run my tests on a clean CouchDB database (without any design documents) / Macbook Pro 2.6Ghz i7, 16GB RAM, SSD / CouchDB 1.4.0

Test script:

var nano = require('nano')('http://localhost:5984');
var async = require('async');
var db = nano.db.use('test');
var mysql      = require('mysql');
var connection = mysql.createConnection({
  host     : 'localhost',
  user     : 'root',
  password : '',
  database: 'test'
});

/*
connection.connect(function(err){
    var t = new Date().getTime() + 10000;
    var i = 0;

    var page = 2,
        lastPage = 100;

    async.whilst(function () {
      return new Date().getTime()  < t;
    },
    function (next) {
        connection.query('INSERT INTO test (firstName, lastName) VALUES ("Testing 001","Testing 002")', function(err, rows, fields) {
          i += 1;
          next();
        });
    },
    function (err) {
        console.log( i );
        connection.end();
    });
});
*/

var t = new Date().getTime() + 10000;
var i = 0;

var page = 2,
    lastPage = 100;

async.whilst(function () {
  return new Date().getTime()  < t;
},
function (next) {
  db.insert({firstName: "Testing 001", lastName: "Testing 002"}, 'id-' + i, function(){
    i += 1;
    next();
  });
},
function (err) {
    console.log( i );
    connection.end();
});

// EDIT:

As it turned out the problem is not on the CouchDB side per say. There is something about client's library/drivers which make them terrible slow. A simple POST test with apache benchmark displays a very good results on the CouchDB side:

$ ab -n 10000 -c 100 -p post-data -T "application/json" "http://192.168.50.102:5984/test/"
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 192.168.50.102 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        CouchDB/1.5.0
Server Hostname:        192.168.50.102
Server Port:            5984

Document Path:          /test/
Document Length:        95 bytes

Concurrency Level:      100
Time taken for tests:   1.149 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      4120412 bytes
Total POSTed:           1920960
HTML transferred:       950095 bytes
Requests per second:    8704.85 [#/sec] (mean)
Time per request:       11.488 [ms] (mean)
Time per request:       0.115 [ms] (mean, across all concurrent requests)
Transfer rate:          3502.69 [Kbytes/sec] received
                        1632.98 kb/s sent
                        5135.67 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       2
Processing:     6   11   2.6     11      23
Waiting:        6   11   2.6     11      22
Total:          6   11   2.6     11      25

Upvotes: 1

Views: 1797

Answers (1)

djc
djc

Reputation: 11711

Are you inserting a single document at a time? You should definitely use the bulk docs loading facility to make a realistic comparison:

http://docs.couchdb.org/en/latest/api/database/bulk-api.html#db-bulk-docs

Read more about CouchDB performance here:

http://guide.couchdb.org/draft/performance.html (a little outdated, but mostly still relevant)

Upvotes: 4

Related Questions