CODEWITHSUNDEEP
node.jsmongodbmongodb-querybigdata
Mikhail Krivosheev
Mikhail Krivosheev

Reputation: 569

How to speed up query response time?

There is a collection in the database, which has about 4.6 million documents.
(data structure can be seen in the screenshot.)

введите сюда описание изображения


When trying to pull at least one value from this collection, the response time to the request reaches more than 3 seconds:
(And this is on my local powerful machine.)

введите сюда описание изображения


Work with the code request and response looks like this:

    // server.js
    const http = require('http');
    const { getProducts } = require('./controllers/currencyController');
    
    const server = http.createServer((req, res) => {
        if(req.url.match(/\/api\/currency.+/g) && req.method === 'GET') {
            getProducts(req, res);
        } else {
            res.writeHead(404, { 'Content-Type': 'application/json' })
            res.end(JSON.stringify({ message: 'Route Not Found' }))
        }
    })
    
    const PORT =  process.env.PORT || 5000
    server.listen(PORT, () => console.log(`Server running on port ${PORT}`))
    module.exports = server;


    //  currencyModel.js
    const {MongoClient} = require('mongodb');
    const client = new MongoClient('mongodb://127.0.0.1:27017/');
    
    const findAll = async (currencyPair) => {
        try{
            await client.connect();
            const testingData = client.db('Trading').collection(currencyPair);
            return testingData;
        }
        catch(e){
            console.log(e);
        }
    }

    module.exports = {
        findAll,
    }


// currencyController.js
const currencyData = require('../models/currencyModel')

async function getProducts(req, res) {
    try {
        const currency_data = await currencyData.findAll('EUR/USD');
        res.writeHead(200, { 'Content-Type': 'application/json' });
        res.end(JSON.stringify(await currency_data.find({id:1}).toArray()));
    } catch (error) {
        console.log(error)
    }
}

module.exports = {
    getProducts,
}

I suspect that the reason for such a long answer may be somehow connected with the lack of indexes. (those numbers that are present are custom created.)

Question:
How can I speed up the response time from the server after receiving the request?
(preferably without using ORM and frameworks.)
And what should ideally be this time?

Upvotes: 0

Views: 883

Answers (1)

Jp Lorandi
Jp Lorandi

Reputation: 113

I'm editing my answer as per the insight gained though the comments.

The findAll method there is a bit of a misnomer, as it just is selecting a collection via the MongoDB driver. The problem is that you are triggering a table scan, that is, MongoDB has to go through every row in the database to see if it matches your expression id:1.

You can cut down the execution time by using findOne instead of find. Once a single record is found that satisfies the expression, it will return it as a result. This is usually not enough, as if the matching record is the last in the database, the performance will be the same as for find.

You should instead create an index:

db.collection.createIndex({"id": 1})

Run this in the MongoDB console, or use MongoDB Compass to create an index. The example creates an index for queries using the id field in ascending order.

If you wanted to query for two fields, say vol and close, vol ascending and close descending, you would create an index via:

db.collection.createIndex({"vol": 1, "close":-1})

Upvotes: 1

Related Questions