Firebase DB Architecting: filter based on two criteria, favorites and location

Question

In my iOS app I'm using Firebase Realtime Database and I have the following models: Users and Venues.

So Users can favorite a Venue. What I want to do is be able to see all the Venues around me that people I follow have favorited. My main issue is I could generate a ‘favoritedFeed’ similar to how FriendlyPix does it, but I really need to consider the distance I am away from the venues and exclude ones that are too far.

Would it be beneficial to just build a ‘favoritedFeed’ of all the venues people I follow favorite, and then filter that feed by distance on the client side? Also I would like to paginate the feed if possible, but if I take this approach, that might not be possible.

I’ll draw my JSON tree, maybe that should be restructured.

users
   [user_id] 
      …
      comments
         [comment_id]  = true
      favorites
         [venue_id] = true
comments
   [comment_id]
      user = [user_id]
      comment = “…..”
venues
  [venue_id]
      ...
      favorites
         [user_id] = true

Right now I just iterate over all of the Users a user follows and then fetch their comments. That way doesn’t scale very well, but its the only way I can figure out how to get the comments at the moment.

Answer 1

You can maintain another root node for the Geofire locations of the events the users you're following have favourited. So whenever someone you follow favourites an event, using Geofire you must write the location of the venue to user-feed-location/$userId for the key $venueId.

I would recommend using Cloud Functions to create an onWrite listener for user-favourites/$userId/$venueId. When a user favourites a venue, for each of their followers, using Geofire update user-feed-location/$followerId with the venue's location and id.

"user-favourites": {
    "$userId": {
        "$venueId": { ... }
    }
},
"user-feed-location": {
    "$userId": {
        "$venueId": {
            "g": "xxxxxxxxxx",
            "l": { ... }
        }
    }
},
"user-followers": {
    "$userId": {
        "$followerId": { ... }
    }
},
"user-followees": {
    "$userId": {
        "$followeeId": { ... }
    }
},
"users": {
    "$userId": { ... }
},
"venue-location": {
    "$venueId": {
        "g": "xxxxxxxxxx",
        "l": { ... }
    }
},
"venues": {
    "$venueId": { ... }
}

By having corresponding Geofire listeners, the addition of a venue to the feed of a particular user within the queried radius would trigger the .keyEntered event. The response block gives you the id and location of the venue with which you can get the venue information and determine the distance between the user and the location returned in the block.

Answer 2

So there are a few different sound approaches you can play with. Depending on your use case, any of them might be the most optimized solution.

Store favorites in the venues and query there

Pros: Very simple to implement and effective to high volumes.

Cons: Might slow down if you have tens of thousands of favorites on each of tens of thousands of venues and have each client performing a very large number of queries against this list. That's probably an edge case though, and you can start here and easily adapt to other models as your app grows.

This approach would use a data structure similar to this:

venues/$venueid/favoritedby/$userid/<true>
venues/$venueid/meta/<detailed data about the venue could be stored here>
users/$userid/myfriends/$userid/<any data or just a true value here>

Now fetching a list of venues requires that I do two things: Find venues favorited by my friends, and filter those based on proximity. I'll cover the proximity separately below and focus on how you get your friends' favorites here.

const ref = firebase.database().ref();
getMyFriends("kato")
  .then(getFavoritesOfMyFriends)
  .then(favorites => console.log(favorites));

function getMyFriends(myUserId) {
  // find everyone I've marked as a friend
  return ref.child(`users/${myUserId}/myfriends`)
    .once('value').then(snap => {
      // return the uids of my friends as an array
      return Object.keys( snap.val() || {} );
    });
}

function getFavoritesOfFriends(myListOfFriends) {
  const promises = [];
  // fetch favorites for each of my friends in parallel
  // and merge them into one object, data will be deduped
  // implicitly since we are storing in a hash
  let mergedFavorites = {};
  myListOfFriends.forEach(uid => {
    promises.push( 
      getFavorites(uid)
        .then(favs => mergeInto(mergedFavorites, favs))
    );
  });
  return Promise.all(promises).then(() => mergedFavorites);
}

function getFavorites(uid) {
  // fetch directly from venues
  return ref.child('venues')
    // but only get items favorited by this uid
    .orderByChild(`favoritedby/${uid}`).equalTo(true)
    .once('value')
    // extract the data from snapshot before returning
    .then(snap => snap.val() || {});
}

function mergeInto(dest, newData) {
  Object.keys(newData).forEach(k => dest[k] = newData[k]);
  return dest;
}

Use a flattened approach

Pros: highly flexible and works with millions of favorites at no real performance penalty

Cons: a bit more complex to implement--quite a bit of hoop-jumping to grab all the data

This approach would use a data structure similar to this:

venues/$venueid/<detailed data about the venue is stored here>
users/$userid/myfriends/$userid/<any data or just a true value here>
favorites/$userid/$venueid/<any data or just a true value here>

Now I can grab my list of friends, and find all their favorite venue ids with something like the following. Then I can look up the individual venues to get names/et al or filter that according to my location.

const ref = firebase.database().ref();

getMyFriends("kato").then(getFavoritesOfMyFriends).then(favorites => console.log(favorites));

function getMyFriends(myUserId) {
  // find everyone I've marked as a friend
  return ref.child(`users/${myUserId}/myfriends`).once('value').then(snap => {
    // return the uids of my friends as an array
    return Object.keys( snap.val() || {} );
  });
}

function getFavoritesOfFriends(myListOfFriends) {
  const promises = [];
  // fetch favorites for each of my friends in parallel
  // and merge them into one object, data will be deduped
  // implicitly since we are storing in a hash
  let mergedVenues = {};
  myListOfFriends.forEach(uid => {
    promises.push(
      getFavorites(uid)
        .then(getVenuesByKey)
        .then(venues => mergeInto(mergedVenues, venues))
    );
  });
  return Promise.all(promises).then(() => mergedVenues);
}

function getFavorites(uid) {
  return ref.child(`favorites/${uid}`).once('value').then(snap => {
    return Object.keys(snap.val()||{});
  });
}

function getVenuesByKey(listOfKeys) {
  const promises = [];
  const venues = {};
  listOfKeys.forEach(venueId => {
    promises.push(
      getVenue(venueId)
        // add each entry to venues asynchronously
        .then(v => venues[venueId] = v)
    );
  });
  // Wait for all items to load then return the compiled list
  return Promise.all(promises).then(() => venues);
}

function getVenue(venueId) {
  return ref.child(`venues/${venueId}`).then(snap => snap.val() || {});
}

function mergeInto(dest, newData) {
  Object.keys(newData).forEach(k => dest[k] = newData[k]);
  return dest;
}

For massive scale

Mostly for others reading this, and for posterity: Note that if you're running a twitter-like feed where a celebrity might have a million followers or similar scenarios, and you really need scale here, you would probably integrate Functions and use them similar to a stored procedure, actually writing duplicate information about the favorite venues out to each follow follower's "feed" for improved scale. But again, you don't need this in most apps, so you can keep it simple.

This is by far the most complex and scalable--it gets hairy when you start editing and changing venue data and has lots of edge cases. Highly recommend becoming intimate with NoSQL data structures before trying something like this.

Filtering by geo

I'd play with these to see how to filter data best locally, but I suspect that you'll end up with something like this:

• Grab the list of venues close to me (see GeoFire)
• Grab the favorites of my friends
• Do something like venuesCloseToMe.some(key => return favoritesOfMyFriends.indexOf(key) > -1);
• Now fetch those venues by key for display on the client