CODEWITHSUNDEEP
mysqlsharding
TommyBs
TommyBs

Reputation: 9648

Scaling Pinterest User tables sharding and ensuring consistency when opening new shards

So this is very much a conceptual question (as much as I'd love to build a billion user app I don't think it's going to happen).

I've read the article by Pinterest on how they scaled their MySQL fleet a number of times ( https://medium.com/@Pinterest_Engineering/sharding-pinterest-how-we-scaled-our-mysql-fleet-3f341e96ca6f ) and I still don't get how they would "open up new shards" without effecting existing users.

The article states that every table is on every shard, including the User table. So I'm assuming that when a user registers and they are assigned a random shard, this has to be done via a function that will always return the same result regardless of the number of shards.

e.g if I sign up with [email protected] they would potentially use that email to work out the shard id and this would have to take into consideration the number of currently 'open' shards. My initial assumption was that they would use something like the mod shard they mentioned later on in the article e.g.

  md5($email) % number_of_shards

But as they open up the number of shards it would change the function result.

I then thought perhaps they had a separate DB to hold purely user info for authentication purposes and this would also contain a column with the assigned shard_id, but as I say the article implies that even the user table is on each shard.

Does anyone else have any ideas or insights into how something like this might work?

Upvotes: 0

Views: 287

Answers (1)

Rick James
Rick James

Reputation: 142453

You are sharding on "user", correct? I see 3 general ways to split up the users.

The modulo approach to sharding has a big problem. When you add a shard, suddenly most users need to move most users to a different shard.

At the other extreme (from modulo) is the "dictionary" approach. You have some kind of lookup that says which shard each user is on. With millions of users, maintenance of the dictionary becomes a costly headache.

I prefer a hybrid:

  1. Do modulo 4096 (or some suitably large number)
  2. Use a dictionary with 4096 entries. This maps 4096 values into the current number of shards.
  3. You have a package to migrate users from one shard to another. (This is a vital component of the system -- you will use it for upgrading, serious crashes, etc, load balancing, etc)
  4. Adding a shard involves moving a few of the 4096 to the new shard and changing the dictionary. The users to move would probably come from the 'busiest' shards, thereby relieving the pressure on them.

Yes, item 4 impacts some users, but only a small percentage of them. You can soften the blow by picking 'idle' or 'small' or 'asleep' users to move. This would involve computing some metric for each of the 4096 clumps.

Upvotes: 1

Related Questions