CODEWITHSUNDEEP
scalafuture
krl
krl

Reputation: 5296

Scala Future or other way to execute Java void methods in parallel

I have the following piece of code:

stringList map copyURLToFile // List[String]

def copyURLToFile(symbol: String) = {
  val url = new URL(base.replace("XXX", symbol))
  val file = new File(prefix + symbol + ".csv")
  org.apache.commons.io.FileUtils.copyURLToFile(url, file)
}

How do I make copyURLToFile execute in parallel?

Scala Futures need a return type, but if I make copyURLToFile of type Future[Unit], what and how do I return from the function?

Upvotes: 1

Views: 658

Answers (2)

Maurício Linhares
Maurício Linhares

Reputation: 40313

You need to return something, how are you going to know which files where downloaded?

Change your code to be:

def copyURLToFile(symbol: String): (String,String) = {
  val url = new URL(base.replace("XXX", symbol))
  val file = new File(prefix + symbol + ".csv")
  org.apache.commons.io.FileUtils.copyURLToFile(url, file)
  return (symbol, file)
}

And then the resulting collection will contain the symbols an the paths to the file where they were downloaded.

Upvotes: 0

Akos Krivachy
Akos Krivachy

Reputation: 4966

If you want a very easy way to parallelize this, then just map over the list using the parallel collections library.

stringList.par.map(copyURLToFile).toList

Upvotes: 3

Related Questions