Json serialization, deserialization in Scala with collections and circular dependencies

Question

So I have been trying to write a JSON parser in Scala. I have so far tried Jackson, gson and flexjson but I can't get it to work with my example included. The example might seem silly, but it demonstrates my problem.

The longest I got was with Jackson using annotations.
@JsonIdentityInfo(generator = ObjectIdGenerators.UUIDGenerator.class)
on every class that I wanted to save. That seemed to create a correct JSON file, but I couldn't deserialize it back to my Garage object. One problem with this approach is also the annotations, if possibly I would like to skip the annotations, since I don't have full control of the source in my real example.

I have inserted all the code (Jackson-example) and my dependencies (in gradle-format) below.

Code:

import java.io.StringWriter

import com.fasterxml.jackson.annotation.{JsonIdentityInfo, ObjectIdGenerators}
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule

object JsonTester extends App {

  trait TestData {
    var volvo1 = new Car(null, "volvo")
    var volvo2 = new Car(null, "volvo")
    var bmw = new Car(null, "bmw")
    var jeep1 = new Car(null, "jeep")
    var jeep2 = new Car(null, "jeep")
    var ford = new Car(null, "ford")

    val p1 = new Person("John", List[Car](volvo1, jeep1))
    volvo1.owner = p1
    jeep1.owner = p1

    val p2 = new Person("Anna", List[Car](volvo2))
    volvo2.owner = p2

    val p3 = new Person("Maria", List[Car](bmw))
    bmw.owner = p3

    val p4 = new Person("Kevin", List(ford, jeep2))
    ford.owner = p4
    jeep2.owner = p4

    val customers = List(p1, p2, p3, p4)
    val carModels = Map("volvo" -> List(volvo1, volvo2), "bmw" -> List(bmw), "jeep" -> List(jeep1, jeep2), "ford" -> List(ford))
    val garage = new Garage[Person, Car]("FixYourCar", customers, carModels);

  }

  new TestData() {
    val originalToString = garage.toString
    println("Garage: " + originalToString)

    val json: String = toJson(garage)
    println(json)
    val garageFromJson: Garage[Person, Car] = fromJson(json)
    println("garageFromJson: " + garageFromJson)

    garageFromJson.customers.foreach(println(_))
    assert(originalToString.equals(garageFromJson.toString))
  }

  def toJson(garage: Garage[Person, Car]): String = {
    import com.fasterxml.jackson.module.scala.DefaultScalaModule

    val mapper = new ObjectMapper()
    mapper.registerModule(DefaultScalaModule)

    println("Saving graph to json")
    val writer = new StringWriter()
    mapper.writeValue(writer, garage)
    writer.toString
  }

  def fromJson(json: String): Garage[Person, Car] = {
    val mapper = new ObjectMapper()
    mapper.registerModule(DefaultScalaModule)
    mapper.readValue[Garage[Person, Car]](json, classOf[Garage[Person, Car]])
  }


}

@JsonIdentityInfo(generator = classOf[ObjectIdGenerators.UUIDGenerator])
case class Garage[P, C](name: String, customers: List[P], models: Map[String, List[C]])

@JsonIdentityInfo(generator = classOf[ObjectIdGenerators.UUIDGenerator])
case class Person(name: String, cars: List[Car])

@JsonIdentityInfo(generator = classOf[ObjectIdGenerators.UUIDGenerator])
case class Car(var owner: Person, model: String) {
  override def toString(): String = s"model: $model, owner:${owner.name}"
}

Dependencies: compile 'org.scala-lang:scala-library:2.11.2' compile "org.scalatest:scalatest_2.11:2.2.2"

compile 'com.typesafe.akka:akka-actor_2.11:2.3.6'
compile 'com.typesafe.akka:akka-testkit_2.11:2.3.6'

compile 'net.sf.opencsv:opencsv:2.3'

compile 'jfree:jcommon:1.0.16'
compile 'org.jfree:jfreechart:1.0.15'
compile 'org.jgrapht:jgrapht-ext:0.9.0'

compile 'org.hibernate:hibernate-core:3.6.0.Final'
compile 'org.hibernate:hibernate-entitymanager:3.6.0.Final'
compile 'mysql:mysql-connector-java:5.1.27'

// json
compile 'com.fasterxml.jackson.module:jackson-module-scala_2.11:2.4.2'
compile 'com.google.code.gson:gson:2.3'
compile 'net.sf.flexjson:flexjson:3.2'

Result from running:

Garage: .....
Saving graph to json
{"@id":"282559ae-70ea-4d74-8363-4b37f1691dba"....
garageFromJson: Garage(FixYourCar,List(Map(@id -> 7ae4b765-c0dc-4a8e-867f-23bc7672db91, name -> John, cars -> ....
Exception in thread "main" java.lang.ExceptionInInitializerError
    at JsonTester.main(JsonTester.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Caused by: java.lang.ClassCastException: scala.collection.immutable.Map$Map3 cannot be cast to Person
    at JsonTester$$anon$1$$anonfun$1.apply(JsonTester.scala:49)
    at scala.collection.immutable.List.foreach(List.scala:381)
    at JsonTester$$anon$1.<init>(JsonTester.scala:49)
    at JsonTester$.<init>(JsonTester.scala:40)
    at JsonTester$.<clinit>(JsonTester.scala)
    ... 6 more

Answer 1

Are you sure you want to actually explicitly put your circular refrences into your serialized form? You do not gain any information by doing so and I can imagine you'll run into a lot of problems. The reason you are not gaining any information is, that, if you know the cars that a person owns, then you can deduce the owner of each car.

In the following, I will reduce your example to only the Car and the Person (i.e., I will omit the Garage), because this scenario is already complicated. I will also not explicitly put the owner of a Car into its serialized form, but I will show you how you could deserialize it and obtain the circular dependencies.

I will use json4s in the example, because I am a bit familiar with it and because I heard it is the de-facto standard for json serialization/deserialization in Scala. You'll also not have to write those nasty annotations you don't like.

The serialization form

As I mentioned, I would not use circular dependencies in your serialization form (though I am sure you could do this somehow by writing a custom serializer/deserializer). Let us imagine the initialization phase of a Car and a Person as a real person going to a vendor and buying a car. So we have a CarBuyingPerson, that is about to buy a List of Cars. The cars must not have an owner yet (suppose the vendor does not count as an owner), so we have a CarWithoutOwner. The two case classes would look as follows:

case class CarBuyingPerson(name: String, cars: List[CarWithoutOwner])
case class CarWithoutOwner(model: String)

Now, we can serialize and deserialize those cars and persons:

val volvo1 = new CarWithoutOwner("volvo")
val volvo2 = new CarWithoutOwner("volvo")
val bmw = new CarWithoutOwner("bmw")
val jeep1 = new CarWithoutOwner("jeep")
val jeep2 = new CarWithoutOwner("jeep")
val ford = new CarWithoutOwner("ford")

val p1 = new CarBuyingPerson("John", List[CarWithoutOwner](volvo1, jeep1))
val p2 = new CarBuyingPerson("Anna", List[CarWithoutOwner](volvo2))
val p3 = new CarBuyingPerson("Maria", List[CarWithoutOwner](bmw))
val p4 = new CarBuyingPerson("Kevin", List(ford, jeep2))


def main(args: Array[String]) {
  implicit val formats = Serialization.formats(NoTypeHints)

  val ser = write(List(p1, p2, p3, p4))
  print(pretty(parse(ser)))
  ???
}

So far so good, but we stil want the cars to have an owner. So let us define our internal classes Car and Person that represent our fully initialized objects. However, they depend on each other so we need to somehow instantate each one before the other. I found an other Stack Overflow post that adresses exactly this problem: Scala: circular references in immutable data types?

The idea is to not pass the constructor arguments by value. (I am however, not sure if the correct term is "call by reference", or "call by name"). So, we define our classes as follows:

class Person(name: String, cars: => List[Car]) {
  override def toString = s"Person $name with cars: $cars"
}

class Car(owner: => Person, model: String) {
  // must not create circular toString calls!
  override def toString = s"Car with model: $model"
}

Now we only need to be able to initialize those classes. So let us define a function buyCars that does this:

case class CarBuyingPerson(name: String, cars: List[CarWithoutOwner]) {
  def buyCars: Person = {
    lazy val This: Person = new Person(name, theCars)
    lazy val theCars: List[Car] = cars map {car => new Car(This, car.model)}
    This
  } 
}

By using lazy vals, we can use a val that has not yet been defined, i.e., we can use theCars when instantiating This. This will give you the desired circular data structure.

Let's test this in the main-method:

def main(args: Array[String]) {
  implicit val formats = Serialization.formats(NoTypeHints)

  val ser = write(List(p1, p2, p3, p4))
  print(pretty(parse(ser)))

  println()
  println()

  val deSer = read[List[CarBuyingPerson]](ser)
  val peopleAfterBuyingCar = deSer map {_.buyCars}
  print(peopleAfterBuyingCar)
}

I find those circular dependencies not easy to understand. My advice to you is to first think hard about whether you really need them. Maybe it would be easyier to change your design and make Car unaware of its owner.