CODEWITHSUNDEEP
mongodb
tman
tman

Reputation: 425

MongoDB issue with NumberLong

we are using a script to copy some data from an older mongo version to mongo 3.6. Works great. To speed it up, we decided to multi-thread it. We've broken up the collection by number of records, divided by number of threads we want to run.

So for a 2.5 billion row table, we run 20 threads, and do the following to get the object IDs, and then use a range of the object IDs per thread: (first 2 shown)

db.m_audit_event.find().sort(['_id']).skip(133432757).pretty().limit(1).next()._id;
db.m_audit_event.find().sort(['_id']).skip(266865514).pretty().limit(1).next()._id;

And it will return an Object ID, i.e.

ObjectId("5976e3af2276c74c154b1c67")

This works fine, until we hit a larger number, and then mongo shell returns...

error: { "$err" : "bad skip value in query", "code" : 10105 } at src/mongo/shell/query.js

So I tried to modify the above using NumberLong, like so:

db.m_audit_event.find().sort(['_id']).skip(NumberLong(2268356869)).pretty().limit(1).next()._id;

I also tried with quotes:

db.m_audit_event.find().sort(['_id']).skip(NumberLong("2268356869")).pretty().limit(1).next()._id;

But same result, both attempts return "bad skip value in query"

Any ideas on how to query mongo so it works with larger numbers?

Upvotes: 0

Views: 931

Answers (1)

Valijon
Valijon

Reputation: 13103

$skip takes a positive integer that specifies the maximum number of documents to skip.

https://docs.mongodb.com/manual/reference/operator/aggregation/skip/#pipe._S_skip

Workaround: (We cannot use long to skip documents (> 231 − 1), but we can use MongoDB aggregation with several $skip's)

db.m_audit_event.aggregate([
  {
    $skip: 1000000000
  },
  {
    $skip: 1000000000
  },
  {
    $skip: 268356869
  },
  {
    $limit: 1
  }
]).next()._id

Upvotes: 1

Related Questions