How to parse JSON in Scala using standard Scala classes?

Question

I am using the build in JSON class in Scala 2.8 to parse JSON code. I don't want to use the Liftweb one or any other due to minimizing dependencies.

The way I am doing it seems too imperative, is there a better way to do it?

import scala.util.parsing.json._
...
val json:Option[Any] = JSON.parseFull(jsonString)
val map:Map[String,Any] = json.get.asInstanceOf[Map[String, Any]]
val languages:List[Any] = map.get("languages").get.asInstanceOf[List[Any]]
languages.foreach( langMap => {
val language:Map[String,Any] = langMap.asInstanceOf[Map[String,Any]]
val name:String = language.get("name").get.asInstanceOf[String]
val isActive:Boolean = language.get("is_active").get.asInstanceOf[Boolean]
val completeness:Double = language.get("completeness").get.asInstanceOf[Double]
}

Answer 1

scala.util.parsing.json.JSON is deprecated.

Here is another approach with circe. FYI documentation: https://circe.github.io/circe/cursors.html

Add the dependency in build.sbt, I used scala 2.13.4, note the scala version must align with the library version.

val circeVersion = "0.14.0-M2"

libraryDependencies ++= Seq(
  "io.circe"  %% "circe-core"     % circeVersion,
  "io.circe"  %% "circe-generic"  % circeVersion,
  "io.circe"  %% "circe-parser"   % circeVersion
)

Example 1:

case class Person(name: String, age: Int)

object Main {
  def main(args: Array[String]): Unit = {
    val input =
      """
        |{
        |  "kind": "Listing",
        |  "data": [
        |    {
        |      "name": "Frodo",
        |      "age": 51
        |    },
        |    {
        |      "name": "Bilbo",
        |      "age": 60
        |    }
        |  ]
        |}
        |""".stripMargin

    implicit val decoderPerson: Decoder[Person] = deriveDecoder[Person] // decoder required to parse to custom object

    val parseResult: Json = circe.parser.parse(input).getOrElse(Json.Null)
    val data: ACursor = parseResult.hcursor.downField("data") // get the data field
    val personList: List[Person] = data.as[List[Person]].getOrElse(null) // parse the dataField to a list of Person
    for {
      person <- personList
    } println(person.name + " is " + person.age)
  }
}

Example 2, json has an object within an object:

case class Person(name: String, age: Int, position: Position)
case class Position(x: Int, y: Int)

object Main {
  def main(args: Array[String]): Unit = {
    val input =
      """
        |{
        |  "kind": "Listing",
        |  "data": [
        |    {
        |      "name": "Frodo",
        |      "age": 51,
        |      "position": {
        |        "x": 10,
        |        "y": 20
        |      }
        |    },
        |    {
        |      "name": "Bilbo",
        |      "age": 60,
        |      "position": {
        |        "x": 75,
        |        "y": 85
        |      }
        |    }
        |  ]
        |}
        |""".stripMargin

    implicit val decoderPosition: Decoder[Position] = deriveDecoder[Position] // must be defined before the Person decoder
    implicit val decoderPerson: Decoder[Person] = deriveDecoder[Person]

    val parseResult = circe.parser.parse(input).getOrElse(Json.Null)
    val data = parseResult.hcursor.downField("data")
    val personList = data.as[List[Person]].getOrElse(null)
    for {
      person <- personList
    } println(person.name + " is " + person.age + " at " + person.position)
  }
}

Answer 2

This is the way I do the Scala Parser Combinator Library:

import scala.util.parsing.combinator._
class ImprovedJsonParser extends JavaTokenParsers {

  def obj: Parser[Map[String, Any]] =
    "{" ~> repsep(member, ",") <~ "}" ^^ (Map() ++ _)

  def array: Parser[List[Any]] =
    "[" ~> repsep(value, ",") <~ "]"

  def member: Parser[(String, Any)] =
    stringLiteral ~ ":" ~ value ^^ { case name ~ ":" ~ value => (name, value) }

  def value: Parser[Any] = (
    obj
      | array
      | stringLiteral
      | floatingPointNumber ^^ (_.toDouble)
      |"true"
      |"false"
    )

}
object ImprovedJsonParserTest extends ImprovedJsonParser {
  def main(args: Array[String]) {
    val jsonString =
    """
      {
        "languages": [{
            "name": "English",
            "is_active": true,
            "completeness": 2.5
        }, {
            "name": "Latin",
            "is_active": false,
            "completeness": 0.9
        }]
      }
    """.stripMargin


    val result = parseAll(value, jsonString)
    println(result)

  }
}

Answer 3

You can do like this! Very easy to parse JSON code :P

package org.sqkb.service.common.bean

import java.text.SimpleDateFormat

import org.json4s
import org.json4s.JValue
import org.json4s.jackson.JsonMethods._
//import org.sqkb.service.common.kit.{IsvCode}

import scala.util.Try

/**
  *
  */
case class Order(log: String) {

  implicit lazy val formats = org.json4s.DefaultFormats

  lazy val json: json4s.JValue = parse(log)

  lazy val create_time: String = (json \ "create_time").extractOrElse("1970-01-01 00:00:00")
  lazy val site_id: String = (json \ "site_id").extractOrElse("")
  lazy val alipay_total_price: Double = (json \ "alipay_total_price").extractOpt[String].filter(_.nonEmpty).getOrElse("0").toDouble
  lazy val gmv: Double = alipay_total_price
  lazy val pub_share_pre_fee: Double = (json \ "pub_share_pre_fee").extractOpt[String].filter(_.nonEmpty).getOrElse("0").toDouble
  lazy val profit: Double = pub_share_pre_fee

  lazy val trade_id: String = (json \ "trade_id").extractOrElse("")
  lazy val unid: Long = Try((json \ "unid").extractOpt[String].filter(_.nonEmpty).get.toLong).getOrElse(0L)
  lazy val cate_id1: Int = (json \ "cate_id").extractOrElse(0)
  lazy val cate_id2: Int = (json \ "subcate_id").extractOrElse(0)
  lazy val cate_id3: Int = (json \ "cate_id3").extractOrElse(0)
  lazy val cate_id4: Int = (json \ "cate_id4").extractOrElse(0)
  lazy val coupon_id: Long = (json \ "coupon_id").extractOrElse(0)

  lazy val platform: Option[String] = Order.siteMap.get(site_id)


  def time_fmt(fmt: String = "yyyy-MM-dd HH:mm:ss"): String = {
    val dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
    val date = dateFormat.parse(this.create_time)
    new SimpleDateFormat(fmt).format(date)
  }

}

Answer 4

This is a solution based on extractors which will do the class cast:

class CC[T] { def unapply(a:Any):Option[T] = Some(a.asInstanceOf[T]) }

object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
object D extends CC[Double]
object B extends CC[Boolean]

val jsonString =
    """
      {
        "languages": [{
            "name": "English",
            "is_active": true,
            "completeness": 2.5
        }, {
            "name": "Latin",
            "is_active": false,
            "completeness": 0.9
        }]
      }
    """.stripMargin

val result = for {
    Some(M(map)) <- List(JSON.parseFull(jsonString))
    L(languages) = map("languages")
    M(language) <- languages
    S(name) = language("name")
    B(active) = language("is_active")
    D(completeness) = language("completeness")
} yield {
    (name, active, completeness)
}

assert( result == List(("English",true,2.5), ("Latin",false,0.9)))

At the start of the for loop I artificially wrap the result in a list so that it yields a list at the end. Then in the rest of the for loop I use the fact that generators (using <-) and value definitions (using =) will make use of the unapply methods.

(Older answer edited away - check edit history if you're curious)

Answer 5

val jsonString =
  """
    |{
    | "languages": [{
    |     "name": "English",
    |     "is_active": true,
    |     "completeness": 2.5
    | }, {
    |     "name": "Latin",
    |     "is_active": false,
    |     "completeness": 0.9
    | }]
    |}
  """.stripMargin

val result = JSON.parseFull(jsonString).map {
  case json: Map[String, List[Map[String, Any]]] =>
    json("languages").map(l => (l("name"), l("is_active"), l("completeness")))
}.get

println(result)

assert( result == List(("English", true, 2.5), ("Latin", false, 0.9)) )

Answer 6

I like @huynhjl's answer, it led me down the right path. However, it isn't great at handling error conditions. If the desired node does not exist, you get a cast exception. I've adapted this slightly to make use of Option to better handle this.

class CC[T] {
  def unapply(a:Option[Any]):Option[T] = if (a.isEmpty) {
    None
  } else {
    Some(a.get.asInstanceOf[T])
  }
}

object M extends CC[Map[String, Any]]
object L extends CC[List[Any]]
object S extends CC[String]
object D extends CC[Double]
object B extends CC[Boolean]

for {
  M(map) <- List(JSON.parseFull(jsonString))
  L(languages) = map.get("languages")
  language <- languages
  M(lang) = Some(language)
  S(name) = lang.get("name")
  B(active) = lang.get("is_active")
  D(completeness) = lang.get("completeness")
} yield {
  (name, active, completeness)
}

Of course, this doesn't handle errors so much as avoid them. This will yield an empty list if any of the json nodes are missing. You can use a match to check for the presence of a node before acting...

for {
  M(map) <- Some(JSON.parseFull(jsonString))
} yield {
  map.get("languages") match {
    case L(languages) => {
      for {
        language <- languages
        M(lang) = Some(language)
        S(name) = lang.get("name")
        B(active) = lang.get("is_active")
        D(completeness) = lang.get("completeness")
      } yield {
        (name, active, completeness)
      }        
    }
    case None => "bad json"
  }
}

Answer 7

This is the way I do the pattern match:

val result = JSON.parseFull(jsonStr)
result match {
  // Matches if jsonStr is valid JSON and represents a Map of Strings to Any
  case Some(map: Map[String, Any]) => println(map)
  case None => println("Parsing failed")
  case other => println("Unknown data structure: " + other)
}

Answer 8

I tried a few things, favouring pattern matching as a way of avoiding casting but ran into trouble with type erasure on the collection types.

The main problem seems to be that the complete type of the parse result mirrors the structure of the JSON data and is either cumbersome or impossible to fully state. I guess that is why Any is used to truncate the type definitions. Using Any leads to the need for casting.

I've hacked something below which is concise but is extremely specific to the JSON data implied by the code in the question. Something more general would be more satisfactory but I'm not sure if it would be very elegant.

implicit def any2string(a: Any)  = a.toString
implicit def any2boolean(a: Any) = a.asInstanceOf[Boolean]
implicit def any2double(a: Any)  = a.asInstanceOf[Double]

case class Language(name: String, isActive: Boolean, completeness: Double)

val languages = JSON.parseFull(jstr) match {
  case Some(x) => {
    val m = x.asInstanceOf[Map[String, List[Map[String, Any]]]]

    m("languages") map {l => Language(l("name"), l("isActive"), l("completeness"))}
  }
  case None => Nil
}

languages foreach {println}