Cryptic Spray Json error message

Question

I'm receiving the following error message when trying to parse some json:

[info]   The future returned an exception of type: spray.httpx.PipelineException, with message: 
Vector("eba760a81b177051b0520418b4e10596955adb98196c15367a2467ab66a19b5c", 600, "AN51SPP6iZBHFJ3aux1jtn6MMMD13Gh3t7", 500,
 ["1BXVXP82f7x9YWdWuCaCYwad8ZoYayyRYt"], "76a91473758c13a91699376abb8fe76931bdd9bdc04ee388ac", false)
 (of class scala.collection.immutable.Vector). (AddressUnspentTXORequestTest.scala:14)

and I'm not really sure what that means. Here is the piece of json that I am trying to parse:

[
  {
    "transaction_hash": "eba760a81b177051b0520418b4e10596955adb98196c15367a2467ab66a19b5c",
    "output_index": 1,
    "value": 600,
    "asset_id": "AN51SPP6iZBHFJ3aux1jtn6MMMD13Gh3t7",
    "asset_quantity": 500,
    "addresses": [
      "1BXVXP82f7x9YWdWuCaCYwad8ZoYayyRYt"
    ],
    "script_hex": "76a91473758c13a91699376abb8fe76931bdd9bdc04ee388ac",
    "spent": false,
    "confirmations": 31674
  },
  {
    "transaction_hash": "1f9f6224bee8813135aba622693c78a33b3460e4efdb340174f87fdd8c9d4148",
    "output_index": 1,
    "value": 600,
    "asset_id": "AS6tDJJ3oWrcE1Kk3T14mD8q6ycHYVzyYQ",
    "asset_quantity": 200000,
    "addresses": [
      "1BXVXP82f7x9YWdWuCaCYwad8ZoYayyRYt"
    ],
    "script_hex": "76a91473758c13a91699376abb8fe76931bdd9bdc04ee388ac",
    "spent": false,
    "confirmations": 35895
  }
]

and here is the case class that I am trying to parse it into:

case class UnspentTXO(transaction_hash: String, output_index: Int, value: Long,
  asset_id: Option[String], asset_quantity: Option[Long], addresses: List[BitcoinAddress],
  script_hex: String, spent: Boolean)

The method initiating the request is here :

  def getUnspentTXOs(address: Address): Future[List[UnspentTXO]] = {
    val pipeline: HttpRequest => Future[List[UnspentTXO]] = 
      sendReceive ~> unmarshal[List[UnspentTXO]]
    pipeline(Get(host + path + address.value + "/unspents"))
  }

and finally this is how I am parsing that Json Request:

override def read(value: JsValue): UnspentTXO = {

  val Seq(transaction_hash, output_index, locked_satoshies, asset_id, asset_quantity, addresses, script_hex, spent) =
    value.asJsObject.getFields("transaction_hash", "value", "asset_id", "asset_quantity", "addresses", "script_hex", "spent")

  val assetId = asset_id match {
    case JsString(s) => Some(s)
    case JsNull => None
    case _ => throw new RuntimeException("Asset id should be of type JsString or JsNull, got something else")
  }

  val assetQuantity = asset_quantity match {
    case JsNumber(n) => Some(n.toLong)
    case JsNull => None
    case _ => throw new RuntimeException("Asset quantity should  be JsNull or a JsNumber")
  }

  // convert JsArray to List[ BitcoinAdress ]
  val addressList = addresses match {
    case ja: JsArray => {
      ja.elements.toList.map( e => BitcoinAddress(e.convertTo[String]))
    }
    case _ => throw new RuntimeException("address list should be of type JsArray, got something else")
  }

  UnspentTXO(transaction_hash.convertTo[String], output_index.convertTo[Int], locked_satoshies.convertTo[Long],
    assetId, assetQuantity, addressList,
    script_hex.convertTo[String], spent.convertTo[Boolean])

}

I think the problem might be that the request is returning a json array instead of just a JSON object, so I am not sure if I am handling that correctly inside of my getUnspentTXOs. The error message seems to be very vague. It seems that spray is trying to wrap the json fields inside of a Vector instead of inside of a UnspentTXO case class. I'm not sure of why this is happening though.

Answer 1

You can not just call convertTo[ Option[ Long ] ] and convertTo[ List [ BitcointAddress ] ].

This is how convertTo is defined,

def convertTo[T :JsonReader]: T = jsonReader[T].read(this)

Which means... only types T for which an implicit evidence of typeclass JsonReader[ T ] is available can be used with convertTo.

Unless you provide appropriate implicit typeclass evidence, you will have to specially handle few cases.

Other than this, spray-json is just too minimalistic... so that so JsObject is just a wrapper on top of Map[ String, JsValue] and getFields is defined as,

 def getFields(fieldNames: String*): immutable.Seq[JsValue] =
   fieldNames.flatMap(fields.get)(collection.breakOut)

Which means... it will just ignore asked fields which are not present in them map, and only return a sequence of JsValue's corresponding to the asked fields which are present.

Hence optional values have to be checked to be present in the map. Or we will have to pattern-match for each possible case ( which can result in a lot of cases).

override def read(value: JsValue): UnspentTXO = {

  val jsObject = value.asJsObject

  // get only non-optional values here 
  val Seq( transaction_hash, output_index, locked_satoshies, addresses,
          script_hex, spent ) =
    jsObject.getFields( "transaction_hash", "output_index", "value", "addresses", "script_hex", "spent" )

  // Assuming you have imported spray.json._, simple types will work.

  // have to handle options differently
  // or 2 optional values would mean 4 patterns-matchings of sequences like above.

  // jsObject.fields is just a Map[ String, JsValue ]

  val assetId = jsObject.fields.get( "asset_id" ) match {
    case Some( JsString( s ) ) => Some( s ) 
    case None => None
  }

  val assetQuantity = jsObject.fields.get( "asset_quantity" ) match {
    case Some( JsNumber( n ) ) => Some( n.toLong ) 
    case None => None
  }

  // convert JsArray to List[ BitcoinAdress ]
  val addressList = addresses match {
    case ja : JsArray => { 
      ja.elements.toList.map( BitcoinAddress( _.convertTo[ String ] ) )
    }
  }

  UnspentTXO( transaction_hash.convertTo[ String ], output_index.convertTo[ Int ],
    locked_satoshies.convertTo[ Long ], assetId, assetQuantity,
    addressList, script_hex.convertTo[ String ], spent.convertTo[ Boolean ] ) 

}

Another way to get convertTo[ Option[ T ] ] and convertTo[ List[ T ] ] working is by importing spray.json.DefaultJsonProtocol._ which provides json-formats for most generally used types. But even then, you must have an implicit evidence of typeclass JsonReader[ T ] in your scope.