CODEWITHSUNDEEP
mongodb
PHAO THU
PHAO THU

Reputation: 135

Can not query TimeStamp oplog.rs by Robomongo tools

In oplog.rs collections have somethings like below:

{
    "ts" : Timestamp(1401265282, 41),
    "h" : NumberLong(-8979599167307291610),
    "v" : 2,
    "op" : "i",
    "ns" : "test",
    "o" : {
        ...........
    }
}

Use Robomongo tool I type the following query:

db.oplog.rs.find({"ts": Timestamp(1401265282,41)})

I get nothing :(

When I use the mongo client tool in console it works.

So is there some things wrong with the Robomongo tools ? I want to use this tool for manage our data but stuck here.

Upvotes: 2

Views: 2372

Answers (2)

Stennie
Stennie

Reputation: 65403

This issue was only recently reported to the Robomongo issue tracker (see: Timestamps being handled incorrectly).

The problem here is actually due to Robomongo 0.8.x using an older version of the mongo shell.

The Timestamp() constructor in the mongo shell had a breaking change in MongoDB 2.4: SERVER-7718: Timestamp constructor in shell should take seconds instead of milliseconds.

I'm not aware of a workaround at the moment (short of using the normal mongo shell to query) since the Timestamp() implementation is native code rather than JavaScript. This issue will be addressed in the next major release of Robomongo (0.9.0 milestone) but there is no ETA for that yet.

Upvotes: 1

John Petrone
John Petrone

Reputation: 27515

Here's the likely issue (I've recreated it on my own copy of Robomongo). I can query running db['oplog.rs'].find() or db['oplog.rs'].findOne() in Robomongo and have no issues. But when I query specifying the the "ts" field in the query it runs a long time and then returns nothing.

The oplog.rs collection is a special capped collection. Capped in that it automatically removes the oldest documents when reaching a set size - http://docs.mongodb.org/manual/core/capped-collections/

Note that for capped collections the normal approach is to query by the insertion order, with either the youngest or the oldest document at the top of the sort:

Query a Capped Collection

If you perform a find() on a capped collection with no ordering specified, MongoDB guarantees that the ordering of results is the same as the insertion order.

To retrieve documents in reverse insertion order, issue find() along with the sort() method with the $natural parameter set to -1, as shown in the following example:

db.cappedCollection.find().sort( { $natural: -1 } )

Special in that there are additional restrictions on oplog.rs due to it's role as a system level collection controlling replication. This leads to some additional restrictions I'll discuss a bit more later.

First, though, let's discuss what's going on here. There is no index on the "ts" field in oplog.rs. So this query will take some time to run if you have run replication for any length of time:

By default, the size of the oplog is as follows:

  • For 64-bit Linux, Solaris, FreeBSD, and Windows systems, MongoDB allocates 5% of the available free disk space, but will always
    allocate at least 1 gigabyte and never more than 50 gigabytes.
  • For 64-bit OS X systems, MongoDB allocates 183 megabytes of space to the oplog.
  • For 32-bit systems, MongoDB allocates about 48 megabytes of space to the oplog.

So that query can run a long time. Also, by the time the query completes, your document may have already been deleted. Why? because the oplog.rs is going to delete the oldest documents to keep under the capped size. Yet if you do a findOne() to pull an example document out it will likely sort it to pull out the oldest document in the collection - which is possibly seconds away from being deleted. You can try this yourself by running findOne() repeatedly on oplog.rs on a busy system - you'll keep getting a different document each time.

Some people have previously tried to address this by creating an index on the "ts" field in oplog.rs but it has not worked. The reason is due to the special nature of oplog.rs - the index will create but it will not be updated. For more details see discussion here: Index on ts field in oplog.rs is not updated

Overall the problem may be worse with Robomongo (it's a bit light on error messages and not clear when it's timing out) but the root cause lies deeper in the MongoDB architecture.

Upvotes: 6

Related Questions