Reputation: 40319

Are there clusters available to rent?

I am wondering if there are clusters available to rent.

Scenario:

We have a program that will take what we estimate a week to run(after optimization) on a given file. Quite possibly, longer. Unfortunately, we also need to do approximately 300+ different files, resulting in approximately 300 weeks of compute time(roundable to 6 wallclock years of continuously running job). For a research job that should be done - at the latest - by December, that's simply unacceptable. While we are exploring other options, I am investigating the option of simply renting a Beowulf cluster. The job is academic and will lead towards the completion of a PhD.

What would be ideal would be a company that we send the source and the job files to the company and then receive a week or two later the result files. Voila!

Quick googling doesn't turn up anything terribly promising.

Answers (10)

Cyril

Reputation: 41

Go to : http://www.extremefactory.com/index.php True HPC cluster, up to 200 TFlops.

Upvotes: 0

Sammy

Reputation: 11

Or you could rent CPU time from a private provider.
I'm from Slovenia and, for example, here we have a great private provider called Arctur. The guys were helpful and and responsive when I contacted them.

You can find them here: hpc.arctur.net

Upvotes: 1

powerrox

Reputation: 1342

There are several ways to get time on clusters.

Purchase time on Amazon elastic cloud. Depending on how familiar you are with their service, it may take time to get it configured the way you want it.
Approach a university and see if they have a commercial program to rent out the time to companies. I know several do. One that I know of specifically is private sector program at NCSA at UIUC. Depending on the institution, they also offer porting and optimization service for your code.

Upvotes: 1

PA.

Reputation: 29369

For academic/scientific use, there are several public centers offering HPC capability. In Europe, there is DEISA. http://www.deisa.eu/ and DEISA members. There must be similar possibilities in the US, probably thru the NSF.
For commercial use, check IBM Deep Computing On Demand offerings. http://www-03.ibm.com/systems/deepcomputing/cod/

Upvotes: 1

argodev

Reputation: 243

The thread has been replete with pointers to Amazon's EC2 - and correctly so. They are the most mature in this area. Recently, they've released their elastic map-reduce platform which sound similar (although not exactly) like what you are trying to do. Google is not an option for you as their compute model doesn't support the generic compute model you need.

Upvotes: 1

Brian Campbell

Reputation: 333126

Amazon EC2 (Elastic Compute Cloud) sounds like exactly what you're looking for. You can sign up for one or more virtual machines (up to 20 automatically, more if you request permission), starting at $0.10 an hour per VM, plus bandwidth costs (free between EC 2 machines and Amazon's other web services). You can choose between several operating systems (various Linux distributions, OpenSolaris, Windows if you pay extra), and you can use pre-existing machine images or create your own. If you're using all open-source software and don't have much bandwidth costs, it sounds like it would cost you around $5000 to run your job (assuming that your 6 years of compute time was for something comparable to their small instances, with a single virtual CPU).

Once you sign up for the service and get their tools set up, it's pretty easy to get new virtual machines launched. I've even spent the $0.10 to launch a machine for a few minutes just to verify an answer I was giving someone here on StackOverflow; I wanted to check something on Solaris, so I just booted up an instance and had a Solaris VM at my disposal within 5 minutes.

Upvotes: 6

Hejazzman

Reputation: 2152

One option is to rent the virtual resources equivalent of whatever number of PCs you need, and set them up as a cluster, using the Amazon Elastic Compute Cloud.

Setting up a beowulf cluster of those is entirely possible.

Check out this link which provides resources and software to do exactly that.

Upvotes: 0

Monkey

Reputation: 1866

I don't know where are you doing your PhD... Most of the Asian, European, and North American universities have some clusters. You can

meet directly the people at the lab which is in charge of the cluster.
ask your PhD director to arrange that. Maybe he/she have some friends that can handle that.

Also, the classical trick is to use the unused time of the computers of your lab/university... Basically, each computer run a client application that crunch numbers when the computer is not used. See http://boinc.berkeley.edu/

Upvotes: 5