Is there a language that takes advantage of massively parallel computers?

Question

There's a company that have/are developing a very parallel computer called Parallella. It looks like it has lots of potential, but it runs some C style language.

Q. Has anyone written a language specifically to take advantage of massively parallel computers like this?

• Clause 1. It has to be a managed garbage collected language.
• Clause 2. It has to make it very easy to write parallel code without requiring the developer to look after low-level locking.
• Clause 3. Bonus points for functional languages.
• Clause 4. Super bonus points for languages with lambdas.

Answer 1

Surprised nobody has mentioned languages based on the BEAM VM, such as Erlang and Elixir. They are functional, process (VM-level) oriented languages that were specifically made to take advantage of multicore CPUs in a distributed environment.

They have independently GC-ed (VM) processes (no Stop-The-World GC) that run concurrently and the VM itself ensures it utilizes all available CPU cores, so your applications end up being both concurrent and parallel.

BEAM VMs can also talk to each other over the network (forming a cluster), and you can invoke functions on different machines by establishing a connection to them, which is similar to RPC.

Elixir is a newer language that has a syntax inspired by Ruby but is strictly a functional language that aims to apply the benefits of Erlang and its packages and VM (similar to Scala-Java) to more modern requirements (web apps etc).

Answer 2

@High Performance Mark - Python is most definitely not a parallel language - the global interpreter lock (GIL) ensures that all Python threads run sequentially (unless they call system functions, when the GIL is released). The best you can hope for with Python is concurrency. Python is great for orchestrating other programs, which might be where you saw it - the GIL doesn’t apply then.

Neither C++ nor C were designed for parallel programming, but newer versions have cobbled some mechanics together - as has Fortran. Formular Translator, better known as an address space manipulator or C without the safeguards. Seriously, the only reason to use Fortran is because you need to use a Fortran library. These are easily recognizable because they are streaked with the blood, sweat and tears left behind by decades of those that went before. Bulletproof, but at what a cost.

Oh, and I know you weren’t saying this, but C, C++ and Fortran violate Clauses 1 and 2.

Listen to mmachenry: the holy clause, the sainted clause, or, as the Spanish would say, the Santa Clause, is to use a functional language. If Parallela needs to sell to Lawrence Livermore, they can write a binary interface to Fortran with the time they’ll save.

NB: Clojure compiles to Java byte code, so if Clojure fits the profile, Java or Scala may also fit. Both have functional extensions.

Answer 3

There are a definitely languages that have been designed to deal with the rising popularity of parallel computing. Parallel processors have sky rocketed in popularity since the death of Moore's Law. Support for better parallel computing in programming languages has followed quickly in its path.

My personal recommendation would be either Haskell or Clojure. Both are functional languages which have made great strides in parallel and concurrent computing leveraging their functional nature to gain advantages. Haskell has a really nice book called Parallel and Concurrent Programming in Haskell by Simon Marlow. I've read it and it's excellent. Clojure has also been built from the ground up with concurrency in mind. An interesting new player in this space is Julia, but I can't say I know much about it at all.

As for clause 1, I don't know what a managed language means. EDIT: What you're calling a managed language is more commonly called garbage collected language. You might want to use that term to help get more effective answers. Also all the languages I recommended have garbage collection.

As for clause 2, Haskell definitely makes parallel computing fairly automatic without any worrying about low level concepts or locking. There is a simple function called 'par' which allows the programmer to annotate two computations to be executed in parallel. The semantics guarantee that the expressions be evaluated when they're necessary and since the computations are functional they are guaranteed not to interact in non-thread-safe ways.

As for clause 3, you're on the right track to be looking for a functional language. Functional subcomputations have automatic thread safety which pays big dividends when it comes to ensuring parallel execution doesn't cause problems. It can't cause any if the computations are functional.

As for clause 4, good luck finding a functional language that doesn't have lambda ;) EDIT: It's not, strictly speaking, part of the definition of a functional language because there is no formal definition for what a functional programing language is. Informally I think a lot of people would mention it as one of the most important features. Concatenative languages or languages that are based on tacit programming (aka point-free style) can be functional and get away with not having lambda. I wouldn't be surprised if the K language didn't have lambda despite being functional. Also, I know for sure combinatory logic (which is the basis for K) does not have lambda. Though combinatory logic is just a theoretical basis and not a practical programming language.