CODEWITHSUNDEEP
mapreducecluster-computinggoogle-compute-engine
Daniel
Daniel

Reputation: 1919

Can you get a cluster of Google Compute Engine instances that are *physically* local?

Google Compute Engine lets you get a group of instances that are semantically local in the sense that only they can talk to each other and all external access has to go through a firewall etc. If I want to run Map-Reduce or other kinds of cluster jobs that are going to induce high network traffic, then I also want machines that are physically local (say, on the same rack). Looking at the APIs and initial documentation, I don't see any way to request that; does anyone know otherwise?

Upvotes: 3

Views: 446

Answers (1)

Joe Beda
Joe Beda

Reputation: 2761

There is no support in GCE right now for specifying rack locality. However, we built the system to work well in the face of large numbers of instances talking to each other in a fully connected way, as long as they are in the same zone.

This is one of the things that allowed MapR to approach the record for a hadoop terasort. You can see that in action in the video for the Criag Mcluckie's talk from IO: https://developers.google.com/events/io/sessions/gooio2012/302/

The best way to see is to test out your application and see how it works.

Upvotes: 7

Related Questions