CODEWITHSUNDEEP
node.jsmongoose
mustafa shaikh
mustafa shaikh

Reputation: 71

How can I optimize MongoDB queries for a large number of records in development mode?

I have 10000 records in my mongodb, when I am trying to fetching the data and apply some filters on that data and send the response back to the client, it takes 14 or 15 seconds to complete the query and send the response back to the client. How can I optimize the query? I am using mongoosejs library in my node application. Here is my code.

const mongoose = require('mongoose')
const { Schema } = mongoose

const StaffSchema = new Schema({
   deptName: {type:String},
   numberOfMembers: {Number}
})

const CollegeSchema = new Schema({
  collegeId:{type:Number},
  collegeName:{type:Number},
  numberOfDepartments: {type:Number},
  founder: {type:String},
  website: {type:String},
  university: {type:String},
  telephone: {type:Number}
  numberOfCoursesAvailable:{type:Number},
  staff: [StaffSchema]
  country: {type:String}
})

module.exports = mongoose.model('College', CollegeSchema, 'colleges')

I am filtering colleges by keyword which can be university or country.

let totalColleges = []
if(country) {
totalColleges = College.find({ country })
} 
if(university) {
 totalColleges = College.find({ university })
}

to fetch a large amount of data like 10,000 records, it takes a huge amount of time We have an option of clustering in the production mode. Please let me know how can I optimize it in development mode.

Upvotes: 0

Views: 2149

Answers (2)

Vipul Ram
Vipul Ram

Reputation: 654

Generally, I have followed the below points.

  • Minimize the use of populate. If you use populate then use select with it. example:Model.find().populate({path:'xyz',select:' name email'})

  • Use select when necessary. example: Model.find().select(firstname lastname email)

  • Use the aggregate function then use the $project field inside it.

    example:Model.aggregate([{$match:{_id:{$exists:true}}},{$project:{_id:1,name:1,email:1}},])

Upvotes: 0

Mohammed Amir Ansari
Mohammed Amir Ansari

Reputation: 2401

The best you can do is add Indexes on the fields you are using to query more often in your queries.

Further limiting the number of records(Pagination), and/or limiting the fields to return in a document(Projection) can be used to optimize the performance, but these are subjected to your requirement.

Also have a look at the below links:

-> https://docs.mongodb.com/manual/core/query-optimization/

-> https://www.yudiz.com/mongodb-query-optimization-techniques/

hope this helps :)

Upvotes: 2

Related Questions