MongoDB: not all the results are returned from a query, using $geoNear

Question

I got this query :

 exports.search = (req, res) => {

  let lat1 = req.body.lat;
  let lon1 = req.body.lng;
  let page = req.body.page || 1;
  let perPage = req.body.perPage || 10;
  let radius = req.body.radius || 100000; // This is not causing the issue, i can remove it and the issue is still here


  var options = { page: page, limit: perPage, sortBy: { updatedDate: -1 } }

  let match = {}

  var aggregate = null;

  if (lat1 && lon1) {

    aggregate = Tutor.aggregate([
      {
        "$geoNear": {
          "near": {
            "type": "Point",
            "coordinates": [lon1, lat1]
          },
          "distanceField": "distance", // this calculated distance will be compared in next section
          "distanceMultiplier": 0.001,
          "spherical": true,
          "key": "loc",
          "maxDistance": radius
        }
      },
      {
        $match: match
      },
      { "$addFields": { "islt": { "$cond": [{ "$lt": ["$distance", "$range"] }, true, false] } } },
      { "$match": { "islt": true } },
      { "$project": { "islt": 0 } }
    ])
    // .allowDiskUse(true);
  } else {
    aggregate = Tutor.aggregate([
      {
        $match: match
      }
    ]);
  }



  Tutor
    .aggregatePaginate(aggregate, options, function (err, result, pageCount, count) {

      if (err) {
        console.log(err)
        return res.status(400).send(err);
      }
      else {

        var opts = [
          { path: 'levels', select: 'name' },
          { path: 'subjects', select: 'name' },
          { path: 'assos', select: 'name' }
        ];
        Tutor
          .populate(result, opts)
          .then(result2 => {
            return res.send({
              page: page,
              perPage: perPage,
              pageCount: pageCount,
              documentCount: count,
              tutors: result2
            });
          })
          .catch(err => {
            return res.status(400).send(err);
          });
      }
    })
};

The query is supposed to retrieve all the tutors in a given range (which is a field from the tutor model, an integer in km, indicating how far the tutor is willing to move) around a certain location. (lat1, lon1).

The issue is that all the documents are not returned. After many tests, I have noticed that only tutors that are less than approximatively 7.5km away from the location are returned and not the others. Even if the tutor is 10km away and has a range of 15km, he won't be returned as he is farer than 7.5km.

I have tried switching location between two tutors (one that is returned and one that is not but should be) to see if this is the only thing causing the issue and it is. After I switched their location (lng and loc), the one that was returned before is no longer and vice versa.

I really don't get why this is happening.

Also, I know the result size is less than 16MB since I don't get all the results, even with allowDiskUse:true.

If you have any other idea about why I'm not getting all the results, don't hesitate !

Thank you !

PS : this is a part of the tutor model with the concerned fields (loc):

import mongoose from 'mongoose';
import validate from 'mongoose-validator';
import { User } from './user';
import mongooseAggregatePaginate from 'mongoose-aggregate-paginate';

var ObjectId = mongoose.Schema.Types.ObjectId;


var rangeValidator = [
    validate({
        validator: (v) => {
            v.isInteger && v >= 0 && v <= 100;
        },
        message: '{VALUE} is a wrong value for range'
    })
];



var tutorSchema = mongoose.Schema({
    fullName: {
        type: String,
        trim: true,
        minlength: [1, 'Full name can not be empty'],
        required: [true, 'Full name is required']
    },
    location: {
        address_components: [
            {
                long_name: String,
                short_name: String,
                types: String
            }
        ],
        description: String,
        lat: Number,
        lng: Number

    },
    loc: {
        type: { type: String },
        coordinates: []
    },


});

tutorSchema.plugin(mongooseAggregatePaginate);
tutorSchema.index({ "loc": "2dsphere" });
var Tutor = User.discriminator('Tutor', tutorSchema);


module.exports = {
    Tutor
};

The user model is using two indexes. The ID and this one ;

db['system.indexes'].find() Raw Output
{
  "v": 2,
  "key": {
    "loc": "2dsphere"
  },
  "name": "loc_2dsphere",
  "background": true,
  "2dsphereIndexVersion": 3,
  "ns": "verygoodprof.users"
}

Answer 1

You are using spherical: true, having the semantics different from planar geospacial queries.

Also, since you are using a GeoJSON object, instead of plain coordinates, the distance MUST be provided in meters (or in kilometers, since you are using the .001 multiplier)

When using spherical: true, it uses the $nearSphere query: https://docs.mongodb.com/manual/reference/operator/query/nearSphere/#op._S_nearSphere

With spherical: false, mongo uses the $near query: https://docs.mongodb.com/manual/reference/operator/query/near/#op._S_near

Since you are working with Lat/Lng, meaning planar coordinates, you should disable the spherical option.

Answer 2

I also have some similar kind of problem, in my case there is problem with limit

https://docs.mongodb.com/manual/reference/operator/aggregation/geoNear/

by default limit is 100 (Optional. The maximum number of documents to return. The default value is 100).

If you want you can increase the limit. Hope it help

Answer 3

The distance returned by $geoNear in case of "spherical": true is in radians so you need to change the "distanceMultiplier": 6371 to be equal to earth radius to get the distance in Km.