CODEWITHSUNDEEP
phpmysqlcachingmemcachedredis
Jonathan Coe
Jonathan Coe

Reputation: 1485

Stream based website - options for optimization with memcache(d)

This is a bit of a tough question to ask without making it too general or broad, so I apologize in advance if it's too high level for this question & answer format.

I am working on a system that includes an often-updated feed type system (a la 'twitter' style), and a system that ranks items based on user votes. The system is based on a json/restful api written in PHP that accesses a (mostly) innodb mysql database. The database itself is actually quite highly optimized itself, and well normalized. The system also utilizes sphinx for searches, and some external queue & processing systems for data processing.

My question is: does anyone have any suggestions for ways to implement memcache in a system where so much is reliant on database table relationships?

For example: the 'hot list' is based on a rank that is calculated every time someone votes on an item. Someone votes, a new 'rank' is calculated for the item, and stored in the mysql database as an integer.

Would it be more efficient to write something like a cron job to cache this list of entries every few minutes, and thus hit the db less frequently - or - would it make sense to pull an 'id' from the DB, and instead of caching the list, cache the item data and pull the data individually from memory for each item in the list?

The curveball is that for each item, the api call also needs to return whether or not the user has voted on the item or not, which could also be cached, but it's hard to cache something that's existence needs to be checked for.

I'm finding it to be a bit of mental gymnastics trying to determine what data it actually makes sense or is possible to cache when so much is reliant on data that is updated so heavily.

Either way, not sure if this is even the right question to be asking, but I would LOVE some insight from anyone who has a lot of experience dealing with caching for applications that have very few static elements.

As an aside, I have seriously considered looking into using Redis to manage the 'ranked item' feed... ie, instead of hitting the database when the rank updates, instead just updating the redis list. I've only spent a little bit of time working with redis though, so i'm not 100% that it is the best way to deal with this on system that needs to be super scalable.

Upvotes: 1

Views: 203

Answers (1)

Marc Gravell
Marc Gravell

Reputation: 1063619

Redis has something important here: pub/sub. This would allow you to publish things like events, and get immediate notifacation to all listeners (very scalable). It also has various "*ex" operations (atomic, existence testing) - SETEX, SETNX etc. plus atomic list swapping etc.

Personally, I think redis has a lot to offer your scenario.

Upvotes: 1

Related Questions