Rewrite string modifications more functional

Question

I'm reading lines from a file

for (line <- Source.fromFile("test.txt").getLines) {
  ....
}

I basically want to get a list of paragraphs in the end. If a line is empty, that starts as a new paragraph, and I might want to parse some keyword - value pairs in the future.

The text file contains a list of entries like this (or something similar, like an Ini file)

User=Hans
Project=Blow up the moon
The slugs are going to eat the mustard. // multiline possible!
They are sneaky bastards, those slugs. 

User=....

And I basically want to have a List[Project] where Project looks something like

class Project (val User: String, val Name:String, val Desc: String) {}

And the Description is that big chunk of text that doesn't start with a <keyword>=, but can stretch over any number of lines.

I know how to do this in an iterative style. Just do a list of checks for the keywords, and populate an instance of a class, and add it to a list to return later.

But I think it should be possible to do this in proper functional style, possibly with match case, yield and recursion, resulting in a list of objects that have the fields User, Project and so on. The class used is known, as are all the keywords, and the file format is not set in stone either. I'm mostly trying to learn better functional style.

Answer 1

You're obviously building something, so you might want to try... a builder!

Like Jürgen, my first thought was to fold, where you're accumulating a result.

A mutable.Builder does the accumulation mutably, with a collection.generic.CanBuildFrom to indicate the builder to use to make a target collection from a source collection. You keep the mutable thing around just long enough to get a result. So that's my plug for localized mutability. Lest one assume that the path from List[String] to List[Project] is immutable.

To the other fine answers (the ones with non-negative appreciation ratings), I would add that functional style means functional decomposition, and usually small functions.

If you're not using regex parsers, don't neglect regexes in your pattern matches.

And try to spare the dots. In fact, I believe that tomorrow is a Spare the Dots Day, and people with sensitivity to dots are advised to remain indoors.

case class Project(user: String, name: String, description: String)

trait Sample {
  val sample = """
  |User=Hans
  |Project=Blow up the moon
  |The slugs are going to eat the mustard. // multiline possible!
  |They are sneaky bastards, those slugs. 
  |
  |User=Bob
  |I haven't thought up a project name yet.
  |
  |User=Greta
  |Project=Burn the witch
  |It's necessary to escape from the witch before
  |we blow up the moon.  I hope Hans sees it my way.
  |Once we burn the bitch, I mean witch, we can
  |wreak whatever havoc pleases us.
  |""".stripMargin
}

object Test extends App with Sample {
  val kv = "(.*?)=(.*)".r
  def nonnully(s: String) = if (s == null) "" else s + " "
  val empty = Project(null, null, null)
  val (res, dummy) = ((List.empty[Project], empty) /: sample.lines) { (acc, line) =>
    val (sofar, cur) = acc
    line match {
      case kv("User", u)    => (sofar, cur copy (user = u))
      case kv("Project", n) => (sofar, cur copy (name = n))
      case kv(k, _)         => sys error s"Bad keyword $k"
      case x if x.nonEmpty  => (sofar, cur copy (description = s"${nonnully(cur.description)}$x"))
      case _ if cur != empty => (cur :: sofar, empty)
      case _                => (sofar, empty)
    }
  }
  val ps = if (dummy == empty) res.reverse else (dummy :: res).reverse
  Console println ps
}

The match can be mashed this way, too:

  val (res, dummy) = ((List.empty[Project], empty) /: sample.lines) {
    case ((sofar, cur), kv("User", u))     => (sofar, cur copy (user = u))
    case ((sofar, cur), kv("Project", n))  => (sofar, cur copy (name = n))
    case ((sofar, cur), kv(k, _))          => sys error s"Bad keyword $k"
    case ((sofar, cur), x) if x.nonEmpty   => (sofar, cur copy (description = s"${nonnully(cur.description)}$x"))
    case ((sofar, cur), _) if cur != empty => (cur :: sofar, empty)
    case ((sofar, cur), _)                 => (sofar, empty)
  }

Before the fold, it seemed simpler to do paragraphs first. Is that imperative thinking?

object Test0 extends App with Sample {
  def grafs(ss: Iterator[String]): List[List[String]] = {
    val (g, rest) = ss dropWhile (_.isEmpty) span (_.nonEmpty)
    val others = if (rest.nonEmpty) grafs(rest) else Nil
    g.toList :: others
  }
  def toProject(ss: List[String]): Project = {
    var p = Project("", "", "")
    for (line <- ss; parts = line split '=') parts match {
      case Array("User", u)    => p = p.copy(user = u)
      case Array("Project", n) => p = p.copy(name = n)
      case Array(k, _)         => sys error s"Bad keyword $k"
      case Array(text)         => p = p.copy(description = s"${p.description} $text")
    }
    p
  }
  val ps = grafs(sample.lines) map toProject
  Console println ps
}

Answer 2

To answer your question without also tackling keyword parsing, fold over the lines and aggregate lines unless it's an empty one, in which case you start a new empty paragraph.

lines.foldLeft(List("")) { (l, x) => 
    if (x.isEmpty) "" :: l else (l.head + "\n" + x) :: l.tail  
} reverse

You'll notice this has some wrinkles in how it handles zero lines, and multiple and trailing empty lines. Adapt to your needs. Also if you are anal about string concatenations you can collect them in a nested list and flatten in the end (using .map(_.mkString)), this is just to showcase the basic technique of folding a sequence not to a scalar but to a new sequence.

This builds a list in reverse order because list prepend (::) is more efficient than appending to l in each step.

Answer 3

Another possible implementation (since this parser is rather simple), using recursion:

import scala.io.Source
case class Project(user: String, name: String, desc: String)
@scala.annotation.tailrec
def parse(source: Iterator[String], list: List[Project] = Nil): List[Project] = {
  val emptyProject = Project("", "", "")
  @scala.annotation.tailrec
  def parseProject(project: Option[Project] = None): Option[Project] = {
    if(source.hasNext) {
      val line = source.next
      if(!line.isEmpty) {
        val splitted = line.span(_ != '=')
        parseProject(splitted match {
          case (h, t) if h == "User" => project.orElse(Some(emptyProject)).map(_.copy(user = t.drop(1)))
          case (h, t) if h == "Project" => project.orElse(Some(emptyProject)).map(_.copy(name = t.drop(1)))
          case _ => project.orElse(Some(emptyProject)).map(project => project.copy(desc = (if(project.desc.isEmpty) "" else project.desc ++ "\n") ++ line))
        })
      } else project
    } else project
  }

  if(source.hasNext) {
    parse(source, parseProject().map(_ :: list).getOrElse(list))
  } else list.reverse
}

And the test:

object Test {
  def source = Source.fromString("""User=Hans
Project=Blow up the moon
The slugs are going to eat the mustard. // multiline possible!
They are sneaky bastards, those slugs.

User=Plop
Project=SO
Some desc""")

  def test = println(parse(source.getLines))
}

Which gives:

List(Project(Hans,Blow up the moon,The slugs are going to eat the mustard. // multiline possible!
They are sneaky bastards, those slugs.), Project(Plop,SO,Some desc))

Answer 4

You're obviously parsing something, so it might be the time to use... a parser!

Since your language seems to treat line breaks as significant, you will need to refer to this question to tell the parser so.

Apart from that, a rather simple implementation would be

import scala.util.parsing.combinator.RegexParsers

case class Project(user: String, name: String, description: String)

object ProjectParser extends RegexParsers {
  override val whiteSpace = """[ \t]+""".r

  def eol : Parser[String] = """\r?\n""".r

  def user: Parser[String] = "User=" ~> """[^\n]*""".r <~ eol
  def name: Parser[String] = "Project=" ~> """[^\n]*""".r <~ eol
  def description: Parser[String] = repsep("""[^\n]+""".r, eol) ^^ { case l => l.mkString("\n") }
  def project: Parser[Project] = user ~ name ~ description ^^ { case a ~ b ~ c => Project(a, b, c) }
  def projects: Parser[List[Project]] = repsep(project,eol ~ eol)
}

And how to use it:

val sample = """User=foo1
Project=bar1
desc1
desc2
desc3

User=foo
Project=bar
desc4 desc5 desc6
desc7 desc8 desc9"""

import scala.util.parsing.input._
val reader = new CharSequenceReader(sample)
val res = ProjectParser.parseAll(ProjectParser.projects, reader)
if(res.successful) {
    print("Found projects: " + res.get)
} else {
    print(res)
}

Answer 5

class Project (val User: String, val Name:String, val Desc: String) {}
object Project {
  def apply(str: String): Project = {
    val user = somehowFetchUserName(str)
    val name = somehowFetchProjectName(str)
    val desc = somehowFetchDescription(str)
    new Project(user, name, desc)
  }
}

val contents: Array[String] = Source.fromFile("test.txt").mkString.split("\\n\\n")
val list = contents map(Project(_))

will end up with the list of projects.