CODEWITHSUNDEEP
node.jsmongodbmongoose
Randy Hall
Randy Hall

Reputation: 8127

Embed document without knowing _id in Mongoose/Mongodb

I know there are no "joins" in MongoDB. I'm attempting to link a large number of documents to the 40,000+ locations in my locations collection.

My locations collection has custom (read: not under my control) identifiers for locations and their corresponding lat/lng coordinates. 

var Locations = new Schema({
    location_id: String, 
    loc: { //lng, lat: as per mongodb documents
         type: [Number], 
         index: '2d'
    } 
});

There are several collections that have a field referencing this custom identifier to match latitude and longitude. 

var MyCollection = new Schema({
    location: String,
    otherFields: Strings...
});

I'm a little lost on how to best go about this. A lot of posts suggest linking via Schema, but I've only seen that with an Schema.Types.ObjectId. This seems impractical for me because the data I'm importing only have the custom identifier.

Could I perhaps add another field into MyCollection and find the correct _id of the location to link to while I'm uploading data. If so, can someone point me in the right direction for accomplishing this.

Map reduce could be used somehow perhaps? I'm still a bit novice with Mongo.

Tried

I did try loading up the entirety of the location data into a JS object then checking that object against the return object from my other query, injecting the matching location data into my return object. This works but is unbearably slow.

Upvotes: 4

Views: 267

Answers (1)

Laokoon
Laokoon

Reputation: 1291

First, just for the record: MongoDb will still generate an _id property for each object you store.

1. "[...], if the mongod receives a document to insert that does not contain an _id field, mongod will add the _id field that holds an ObjectId. [...]"

Source

You wrote that location_id is not under your control. And you want to use location_id because the other collections are using it also? So you don't want to break the standard in your project which is good.

As I see, you already have the location property in MyCollection and can store the location_id there. As far as I know, you have to write your own linking methods now. You have to store the location_id in MyCollection and load the Location if you want to access if via MyCollection by 

Locations.find({location_id: <the_location_id>})

But maybe your main problem is that you cannot find the Locations you are looking for in a reliable time?

I don't know your criteria for finding Locations for MyCollection is. If it is proximity of coordinates then you can reduce the amount of locations to check by filtering out the ones you really don't need to check. Then you don't have to check all the 40.000 Locations, but maybe just 100? In the following I assume it is the proximity.

Do you have lat, lon in both collections (Locations, MyCollection) ? If so, you can define a query which gets locations around your MyCollection object (square). Then you will receive a smaller amount of Locations from MongoDb. Now you can apply your more complex check which checks if they really belong to your MyCollection-object.

something like this:

Locations.find({lat: {$gt: <x>-a, $lt: <x>+a}, lon: {$gt: <y>-b, $lt: <y>+b}}, function(locations){ ... });

I hope it helps.

Upvotes: 1

Related Questions