CODEWITHSUNDEEP
scalaapache-sparkbigdata
pythonic
pythonic

Reputation: 21605

Simplest form of multithreading in scala

So, I have a portion of code that looks like the following.

for(a <- 0 until segments)
{
    // do something with elements at index a
}

What is the simplest type of multithreading, that can allow me to execute each element of that loop in a separate thread. I also have global collections (indexed by a) that need to be accessed by each thread. All I see on the internet are futures and actors, but they are quite difficult to grasp. I want something as simple as OpenMP.

I tried the following, but it gave me the error, ';' expected but '.' found. }} thread.start.

for (a <- 0 until segments) {
    val thread = new Thread {
        override def run {
            // do something with elements at index a
        }} thread.start
}

Upvotes: 1

Views: 4116

Answers (2)

Łukasz
Łukasz

Reputation: 8663

I asumed you are executing the code just for side effects.

Following code works. You get an error because thread.start should be in a separate line.

for (a <- 0 until 10) {
    val thread = new Thread {
        override def run(): Unit = { 
            // code
        }
    }
    thread.start()
}

you can replace it using futures, to use a thread pool which is much better than spawning possibly a large number of threads.

for (a <- 0 until 10) {
    Future {
        // code
    }
}

however in either case you won't be able to wait till it completes. You can traverse the range and get back the Future that will contains list of all partial results from each of the Futures you start using the second function.

val result: Future[List[Unit]] = Future.traverse((0 until 10).toList)(index => Future {
    // code
})

Having a Future result you can await on it, i.e. block calling thread until all computations are finished. 

import scala.concurrent.duration._
import scala.concurrent.Await
Await.ready(result, 1.hour)

Use Await.result to get the result, Await.ready just waits till future is completed.

To wait for threads you would need to build a list when starting them and then call join on each.

I don't think you would need to change existing code to use parallel collections but maybe I don't know something specific to your code. Following code will split the range into chunks that will be processed in parallel.

for (a <- (0 until 10).par) {
   // code
}

With parallel collections you don't need to wait, thread will be blocked until everything is processed.

Upvotes: 5

Dima
Dima

Reputation: 40500

You are wrong about parallel collections: they won't require you to change any code beyond what you actually want to change:

 (0 until segments).par.foreach { a =>
    // code
 }

Upvotes: 1

Related Questions