CODEWITHSUNDEEP
mongodbsharding
Aaron
Aaron

Reputation: 1009

Restrict querying MongoDB collection to inactive chunks only

I am building an application which will perform 2 phases.

  1. Execute Phase - First phase is very INSERT intensive (as many inserts as the hardware can possibly can execute in a second). This is essentially a logging trail of work performed.
  2. Validation Phase - Next phase will query the logs generated by phase 1 and compare to an external source and perform an UPDATE on the record to store some statistics. This process is second priority to phase 1.

I'm trying to see if its feasible to do them in parallel and keep write locking to a minimum for the execution phase. I thought one way to do this would be to restrict my Validation phase to only query from older records which are not in the chunk currently being inserted to by the execution phase. Is there something in MongoDB that restricts a find() to only query from chunks that have not been accessed in some configurable amount of time?

Upvotes: 1

Views: 131

Answers (2)

Karoly Horvath
Karoly Horvath

Reputation: 96258

You can use the mentioned replica set with slaveOk, and update in the master.

You can use a timestamp field or an ObjectId (which already contains a timestamp) for filtering.

Upvotes: 1

Comtaler
Comtaler

Reputation: 1630

You probably want to set up replica set. Insert into the master and fetch from secondaries. In that way, your insert won't be blocked at all.

Upvotes: 2

Related Questions