CODEWITHSUNDEEP
javascriptnode.jsmultithreadingsqlite
J.J
J.J

Reputation: 3607

Parallel SQLite queries in Node.js

Node.js has some very good bindings for SQLite thanks to the SQLite3 package, but unfortunately, since Node runs single-threaded, all queries are done in the same connection. SQLite runs all queries per connection in series, meaning that it is impossible to actually do parallel queries in SQLite under Node. For more details check this https://github.com/mapbox/node-sqlite3/issues/394

This is true whether you wrap your query-logic in async.each/async.parallel, or any of the other helper packages for parallelising/serialising database IO. At the end of the day, to benefit from Node's async IO architecture under SQLite3, you are going to need more than one processing thread.

How can this be done? :)

Upvotes: 0

Views: 2871

Answers (1)

РАВИ
РАВИ

Reputation: 12155

You can use cluster api to create child processes and inside of those you can run different actions, in your case db queries. https://nodejs.org/api/cluster.html.

Note: How many clusters you want to create depends on your hardware.

const cluster = require('cluster');
const db = new sqlite3.Database(':memory:')

if (cluster.isMaster) {
    console.log(`Master ${process.pid} is running`);
    cluster.fork()
        .on('exit', (worker, code, signal) => {
            console.log(`worker ${worker.process.pid} died`);
        });
    cluster.fork()
        .on('exit', (worker, code, signal) => {
            console.log(`worker ${worker.process.pid} died`);
        });

} else if (cluster.worker.id === 1){
  db.run(QUERY1);
} else {
 db.run(QUERY2);
}

Upvotes: 2

Related Questions