CODEWITHSUNDEEP
scalaioparallel-processingtext-files
elm
elm

Reputation: 20415

Read multiple text files simultaneously in Scala

Given a collection of filenames to be read and processed, namely for instance,

val inFilenames = (1 to 100).map(n => s"file$n.txt")

how to read all 100 text files simultaneously, so that

def processFile[A](inFilename: String, f: => A): Int

may be called up to 100 times in parallel?

Many Thanks

Upvotes: 0

Views: 1501

Answers (1)

Ende Neu
Ende Neu

Reputation: 15783

Seems something you can solve with Futures:

def processFile[A](inFilename: String, f: => A): Int = ???

def someFunction[A]: A = ???

val inFilenames = (1 to 100).map { n => s"file$n.txt" }

val futures: List[Future[Int]] = 
  inFilenames.map(file => Future(processFile(file, someFunction))).toList

This will create a List with 100 Futures. Futures start running as soon as they are declared, to retrieve the results you can add the onCallback method or block (not suggested) using Await.result(futures), in a more compact form it would be:

inFilenames.map(file => Future(processFile(file, someFunction)).onComplete {
  case Success(res) => println(res)
  case Failure(e) => println("failure: " + e.getMessage)
})

Upvotes: 3

Related Questions