CODEWITHSUNDEEP
pythongoogle-app-enginehadoopmapreduce
daydreamer
daydreamer

Reputation: 91999

running hadoop on Google app engine?

Is it possible to run map reduce jobs on Google app engine?

Any reference or tutorial would help

Thanks

Upvotes: 2

Views: 1932

Answers (4)

CHZ
CHZ

Reputation: 1

Saw this Google Cloud Platform advertisement:
Hadoop on Google Compute Engine virtual machines
https://cloud.google.com/solutions/hadoop

Upvotes: 0

Leo Romanovsky
Leo Romanovsky

Reputation: 1790

Here is the full documentation: https://developers.google.com/appengine/docs/python/dataprocessing/overview

Upvotes: 0

18bytes
18bytes

Reputation: 6029

You cannot run Hadoop on Appengine (No filesystem access as well).

You may want to check AWS ElasticMapreduce. Its a cloud based platform for running Mapreduce jobs.

ElasticMapreduce

Upvotes: 0

Daniel Roseman
Daniel Roseman

Reputation: 599630

Sort of.

You can't use the actual MapReduce framework - the architecture is too incompatible with AppEngine.

However, there is an equivalent system built specficially for GAE - appengine-mapreduce. That site is a bit confusing, as the first version of the code only supported mappers, without the subsequent reduce step - recently they released a version with full mapreduce support, but some of the documentation still referes to the earlier mapper-only one.

The best introduction is the GoogleIO talk from Mike Aizatskyi.

Upvotes: 7

Related Questions