Play JSON: Reading and validating a JsObject with unknown keys

Question

I'm reading a nested JSON document using several Reads[T] implementations, however, I'm stuck with the following sub-object:

{
    ...,
    "attributes": {
        "keyA": [1.68, 5.47, 3.57],
        "KeyB": [true],
        "keyC": ["Lorem", "Ipsum"]
     },
     ...
}

The keys ("keyA", "keyB"...) as well as the amount of keys are not known at compile time and can vary. The values of the keys are always JsArray instances, but of different size and type (however, all elements of a particular array must have the same JsValue type).

The Scala representation of one single attribute:

case class Attribute[A](name: String, values: Seq[A])
// 'A' can only be String, Boolean or Double

The goal is to create a Reads[Seq[Attribute]] that can be used for the "attributes"-field when transforming the whole document (remember, "attributes" is just a sub-document).

Then there is a simple map that contains allowed combinations of keys and array types that should be used to validate attributes. Edit: This map is specific for each request (or rather specific for every type of json document). But you can assume that it is always available in the scope.

val required = Map(
  "KeyA" -> "Double",
  "KeyB" -> "String",
  "KeyD" -> "String",
)

So in the case of the JSON shown above, the Reads should create two errors:

1. "keyB" does exist, but has the wrong type (expected String, was boolean).
2. "keyD" is missing (whereas keyC is not needed and can be ignored).

---

I'm having trouble creating the necessary Reads. The first thing I tried as a first step, from the perspective of the outer Reads:

...
(__ \ "attributes").reads[Map[String, JsArray]]...
...

I thought this is a nice first step because if the JSON structure is not an object containing Strings and JsArrays as key-value pairs, then the Reads fails with proper error messages. It works, but: I don't know how to go on from there. Of course I just could create a method that transforms the Map into a Seq[Attribute], but this method somehow should return a JsResult, since there are further validations to do.

The second thing I tried:

  val attributeSeqReads = new Reads[Seq[Attribute]] {
    def reads(json: JsValue) = json match {
      case JsObject(fields) => processAttributes(fields)
      case _ => JsError("attributes not an object")
    }
    def processAttributes(fields: Map[String, JsValue]): JsResult[Seq[Attribute]] = {
      // ...
    }
  }

The idea was to validate each element of the map manually within processAttributes. But I think this is too complicated. Any help is appreciated.

edit for clarification:

At the beginning of the post I said that the keys (keyA, keyB...) are unknown at compile time. Later on I said that those keys are part of the map required which is used for validation. This sounds like a contradiction, but the thing is: required is specific for each document/request and is also not known at compile time. But you don't need to worry about that, just assume that for every request the correct required is already available in the scope.

Answer 1

You are too confused by the task

The keys ("keyA", "keyB"...) as well as the amount of keys are not known at compile time and can vary

So the number of keys and their types are known in advance and the final?

So in the case of the JSON shown above, the Reads should create two errors:

1. "keyB" does exist, but has the wrong type (expected String, was boolean).

2. "keyD" is missing (whereas keyC is not needed and can be ignored).

Your main task is just to check the availability and compliance?

You may implement Reads[Attribute] for every your key with Reads.list(Reads.of[A]) (this Reads will check type and required) and skip omitted (if not required) with Reads.pure(Attribute[A]). Then tuple convert to list (_.productIterator.toList) and you will get Seq[Attribute]

val r = (
  (__ \ "attributes" \ "keyA").read[Attribute[Double]](list(of[Double]).map(Attribute("keyA", _))) and
    (__ \ "attributes" \ "keyB").read[Attribute[Boolean]](list(of[Boolean]).map(Attribute("keyB", _))) and
    ((__ \ "attributes" \ "keyC").read[Attribute[String]](list(of[String]).map(Attribute("keyC", _))) or Reads.pure(Attribute[String]("keyC", List()))) and 
    (__ \ "attributes" \ "keyD").read[Attribute[String]](list(of[String]).map(Attribute("keyD", _)))        
  ).tupled.map(_.productIterator.toList)

scala>json1: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyD":["Lorem","Ipsum"]}}

scala>res37: play.api.libs.json.JsResult[List[Any]] = JsSuccess(List(Attribute(keyA,List(1.68, 5.47, 3.57)), Attribute(KeyB,List(true)), Attribute(keyC,List()), Attribute(KeyD,List(Lorem, Ipsum))),)   

scala>json2: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyC":["Lorem","Ipsum"]}}    

scala>res38: play.api.libs.json.JsResult[List[Any]] = JsError(List((/attributes/keyD,List(ValidationError(List(error.path.missing),WrappedArray())))))    

scala>json3: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":["Lorem"],"keyC":["Lorem","Ipsum"]}}    

scala>res42: play.api.libs.json.JsResult[List[Any]] = JsError(List((/attributes/keyD,List(ValidationError(List(error.path.missing),WrappedArray()))), (/attributes/keyB(0),List(ValidationError(List(error.expected.jsboolean),WrappedArray())))))

If you will have more than 22 attributes, you will have another problem: Tuple with more than 22 properties.

for dynamic properties in runtime

inspired by 'Reads.traversableReads[F[_], A]'

def attributesReads(required: Map[String, String]) = Reads {json =>
  type Errors = Seq[(JsPath, Seq[ValidationError])]

  def locate(e: Errors, idx: Int) = e.map { case (p, valerr) => (JsPath(idx)) ++ p -> valerr }

  required.map{
    case (key, "Double") => (__ \  key).read[Attribute[Double]](list(of[Double]).map(Attribute(key, _))).reads(json)
    case (key, "String") => (__ \ key).read[Attribute[String]](list(of[String]).map(Attribute(key, _))).reads(json)
    case (key, "Boolean") => (__ \ key).read[Attribute[Boolean]](list(of[Boolean]).map(Attribute(key, _))).reads(json)
    case _ => JsError("")
  }.iterator.zipWithIndex.foldLeft(Right(Vector.empty): Either[Errors, Vector[Attribute[_ >: Double with String with Boolean]]]) {
      case (Right(vs), (JsSuccess(v, _), _)) => Right(vs :+ v)
      case (Right(_), (JsError(e), idx)) => Left(locate(e, idx))
      case (Left(e), (_: JsSuccess[_], _)) => Left(e)
      case (Left(e1), (JsError(e2), idx)) => Left(e1 ++ locate(e2, idx))
    }
  .fold(JsError.apply, { res =>
    JsSuccess(res.toList)
  })
}

(__ \ "attributes").read(attributesReads(Map("keyA" -> "Double"))).reads(json)

scala> json: play.api.libs.json.JsValue = {"attributes":{"keyA":[1.68,5.47,3.57],"keyB":[true],"keyD":["Lorem","Ipsum"]}}

scala> res0: play.api.libs.json.JsResult[List[Attribute[_ >: Double with String with Boolean]]] = JsSuccess(List(Attribute(keyA,List(1.68, 5.47, 3.57))),/attributes)