CODEWITHSUNDEEP
javascalaapache-spark
pythonic
pythonic

Reputation: 21589

How to stream stdout in Scala?

Basically, I have a program whose output goes to stdout. I am running that program from within my Scala code. What I do right now is to redirect the output to a file. Something like this.

val cmd = progName + " " + arguments 
cmd #> new java.io.File(outputFilePath) !

I also have to later process that output file. That is why this approach is slow, as I have to first wait for the program to have finished and everything written to the output file. What I would prefer is to have content of the stdout being streamed so that from another thread I could read the contents as they are being streamed. In this way, I can process the data while the program I am running from Scala is running.

Moreover, the data in the output file is delimited by newlines. Is there a way, I could get the data line by line as the program is running.

Lastly, is there something in Java to do that, which I can also then use in Scala?

Upvotes: 3

Views: 9908

Answers (1)

DaoWen
DaoWen

Reputation: 33019

The ProcessBuilder has a lineStream method, which I think does exactly what you want. It returns a Stream[String] that lets you process the output as it becomes available.

Here's a simple program with slow output for testing, which I saved as slow-printer.scala:

// slow-printer.scala
for (i <- 1 to 10) {
  println(i)
  Thread.sleep(1000)
}

You can stream the output (1 line per second) from the program like this:

import scala.sys.process._

val lines = "scala slow-printer.scala".lineStream

lines foreach println

If you're stuck with 2.10, or maybe if your output isn't newline-delimited, then you can use ProcessIO to handle the output instead:

import scala.sys.process._

Seq("scala", "slow-printer.scala").run(new ProcessIO(
  _.close(), // stdin
  out => { // stdout
    val src = scala.io.Source.fromInputStream(out)
    for (line <- src.getLines()) {
      println(line)
    }
  },
  _.close() // stderr
))

Upvotes: 5

Related Questions