CODEWITHSUNDEEP
scalafuture
Bob Dem
Bob Dem

Reputation: 1031

Future call in for comprehension in Scala - how do I process them sequentially?

I have a code similar to this:

for (n <- 1 to 1000) {
  someFuture map {
    // some other stuff
}

This is a basic piece of code and works fine. However, somefuture does some queries to a database, and the database cannot receive several queries in parallel, which is what happens before (it spawns a lot of threads executing somefuture as one would expect).

Ideally, I would like to do it sequentially (i.e. call someFuture when n=1, do some processing, call someFuture when n=2, do some processing, etc). I thought about using some blocking method (from Await) but this happens inside an actor, so blocking is not a good idea. Another idea was creating a fixed thread pool for this particular future call, but sounds like overkill. What should I do instead?

Update: I have found this answer which suggests creating a fixed thread pool as I thought. Still, is this the right way to do it?

Upvotes: 0

Views: 222

Answers (3)

Joe K
Joe K

Reputation: 18434

Probably the ideal long-term way to handle this is to use a database access layer that does connection pooling. Most frameworks like play or slick have some preferred way of handling this, or if you want something standalone, DBCP might be a good option. I think most of these should have a "natural" way to limit the number of connections to a fixed size and block if no connections in the pool are available, which would limit your parallelism.

Other than introducing some other dependency like that, using a thread pool execution context as you mentioned is definitely the way to go. It's not overkill; it's very common, and will be much less hacky than any other way of handling this.

Upvotes: 0

som-snytt
som-snytt

Reputation: 39587

You want to map or flatmap a single future.

scala> val f = Future(42)
f: scala.concurrent.Future[Int] = Future(Success(42))

scala> (1 to 10).foldLeft(f)((f,x) => f.map(i => i + 1))
res1: scala.concurrent.Future[Int] = Future(<not completed>)

scala> res1
res2: scala.concurrent.Future[Int] = Future(Success(52))

scala> (1 to 10).foldLeft(f)((f,i) => {
     | println(i)
     | f.map(x => x+i) })
1
2
3
4
5
6
7
8
9
10
res4: scala.concurrent.Future[Int] = Future(<not completed>)

scala> res4
res5: scala.concurrent.Future[Int] = Future(Success(97))

Upvotes: 1

rogue-one
rogue-one

Reputation: 11587

one approach would be to send the message to an actor that processes the data. Since actor processes messages one by one you would execute your queries sequentially not in parallel. 

for (n <- 1 to 1000) {
  someFuture map {
      x => actor ! x
    }
}

Upvotes: 0

Related Questions