CODEWITHSUNDEEP
scala
Jayadeep Jayaraman
Jayadeep Jayaraman

Reputation: 2825

Scala Parallel Processing

I am writing a code which would take a list of database tables from a metadata repo and move data between source and target database as configured in the metadata repository. Below are my requirements requirements:-

  1. Process each of the tables in the list in parallel and move data between source and target.
  2. Generate a report which will be sent via email which will have information on which tables have been moved successfully and which have failed.

Below is a sample implementation of the code. The method futureCalc will have the implementation of moving the data. Below are my questions:-

1.For sending the email report do I need to use Async.result as a blocking session to get the list of reports or is there a non blocking way to achieve this?

2.I have read in multiple forums that Callbacks should be used instead of a blocking API, are callbacks called asynchronously in a separate thread from the main thread?

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.util.Success
import scala.util.Failure
import scala.async.Async._

object test extends App {
  def futureCalc(i:Int):Int = {
    val sleeptime = i*60
    println("This is start of thread:"+i)
    Thread.sleep(sleeptime)
    println("This is after sleep :"+i)
    if (i<4)
      i*i
    else
      throw new RuntimeException("Bad thing")
  }

  val iniList   = List(1,2,3,4)
  val resultList = iniList map (i => Future{futureCalc(i)})

  resultList map  (i => {while(!i.isCompleted){
    i.onComplete {
      case Success(j) => println("No Issues")
      case Failure(ex) => println("Something went wrong ")
    }
  }
  }
    )

}

Upvotes: 1

Views: 150

Answers (1)

Epicurist
Epicurist

Reputation: 923

If you say Scala and database, use Lightbend's SLICK. The results are futures which can executed like this:

val q2 = for {
  c <- coffees if c.price < 9.0
  s <- suppliers if s.id === c.supID
} yield (c.name, s.name)`

See the Slick documentation.

Upvotes: 1

Related Questions