CODEWITHSUNDEEP
c#mongodbperformancemongodb-query
Joakim
Joakim

Reputation: 2109

MongoDB Count on millions of row is very slow

Using MongoDB 2.4 with mongoDB .NET driver 3.2

I have a collection with 30 million entries.

var visits = new MongoHelper<CampaignVisitLog>()
         .GetCollection().AsQueryable().Count(t => t.campaignId == campaignId);

campaignId is indexed. Depending on many entries campaignId has it will take from 30 seconds to several minutes to return the count.

Whats the correct way to count this collection?

Upvotes: 0

Views: 2515

Answers (3)

ZOXEXIVO
ZOXEXIVO

Reputation: 960

MongoDB C# driver is sick.

LINQ queries are always translated to aggregation framework pipelines.

var pipeline = [ { "$group" : { "_id" : 1, "__result" : { "$sum" : 1 } } }]
db.test.aggregate(pipeline)

This results in a full collection scan on the server because the LINQ query did not specify any constraints.

Upvotes: 1

MementoMori
MementoMori

Reputation: 1

ensure you have enough memory to store your index in ram.

https://docs.mongodb.org/manual/tutorial/ensure-indexes-fit-ram/

Upvotes: 0

Mat&#237;as Fidemraizer
Mat&#237;as Fidemraizer

Reputation: 64923

I have a collection with 30 million entries.

Independently of optimizing the whole query, you can't think that you're going to get blazing-fast responses with millions of items.

If you're performing other queries to get stats, maybe it's the time to schedule this calculations and do them with some asynchronous service (i.e. a Windows service, Windows scheduled task, Quartz.NET...), and get their results also asynchronously.

You can either use MongoDB to store your calculation service results or go for a more specific solution: a service bus (i.e. RabbitMQ, Azure Service Bus, NServiceBus...).

Upvotes: 1

Related Questions