CODEWITHSUNDEEP
scalacirce
Tae
Tae

Reputation: 1695

How to derive a decoder semiautomatically for a list of some type with Circe?

I have an implicit class that decodes server's response into a JSON and latter in the right case class to avoid repeating calls to .as and .getOrElse all around the tests:

implicit class RouteTestResultBody(testResult: RouteTestResult) {
  def body: String = bodyOf(testResult)
  def decodedBody[T](implicit d: Decoder[T]): T =
    decode[Json](body)
      .fold(err => throw new Exception(s"Body is not a valid JSON: $body"), identity)
      .as[T]
      .getOrElse(throw new Exception(s"JSON doesn't have the right shape: $body"))
}

Of course, it relies on us passing a decoder:

import io.circe.generic.semiauto.deriveDecoder

val result: RouteTestResult = ...
result.decodedBody(deriveDecoder[SomeType[AnotherType])

It works most of the time, but fails when the response is a list:

result.dedoceBody(deriveDecoder[List[SomeType]])
// throws "JSON doesn't have the right shape"

How can I semiautomatically derive a decoder for a list with specific types inside?

Upvotes: 2

Views: 2856

Answers (1)

Travis Brown
Travis Brown

Reputation: 139028

The terminology here is unfortunately overloaded, in that we use "deriving" in two senses:

  • Providing an instance for e.g. List[A] given an instance for A.
  • Providing an instance for a case class or sealed trait hierarchy given instances for all member types.

This problem isn't specific to Circe, or even Scala. In writing about Circe I generally try to avoid referring to the first kind of instance generation as "derivation" at all, and to refer to the second kind as "generic derivation" to emphasize that we're generating instances via a generic representation of the algebraic data type.

The fact that we sometimes use the same word to refer to both kinds of type class instance generation is a problem because they're typically very distinct mechanisms in Scala. In Circe the thing that provides an encoder or decoder instance for List[A] given one for A is a method in the type class companion object. For example, in the object Decoder in circe-core we have a method like this:

implicit def decodeList[A](implicit decodeA: Decoder[A]): Decoder[List[A]] = ...

Because this method definition is in the Decoder companion object, if you ask for an implicit Decoder[List[A]] in a context where you have an implicit Decoder[A], the compiler will find and use decodeList. You don't need any imports or extra definitions. For example:

scala> case class Foo(i: Int)
class Foo

scala> import io.circe.Decoder, io.circe.parser
import io.circe.Decoder
import io.circe.parser

scala> implicit val decodeFoo: Decoder[Foo] = Decoder[Int].map(Foo(_))
val decodeFoo: io.circe.Decoder[Foo] = io.circe.Decoder$$anon$1@6e992c05

scala> parser.decode[List[Foo]]("[1, 2, 3]")
val res0: Either[io.circe.Error,List[Foo]] = Right(List(Foo(1), Foo(2), Foo(3)))

If we desugared the implicit machinery here, it would look like this:

scala> parser.decode[List[Foo]]("[1, 2, 3]")(Decoder.decodeList(decodeFoo))
val res1: Either[io.circe.Error,List[Foo]] = Right(List(Foo(1), Foo(2), Foo(3)))

Note that we could replace first kind of derivation with the second, and it would still compile:

scala> import io.circe.generic.semiauto.deriveDecoder
import io.circe.generic.semiauto.deriveDecoder

scala> parser.decode[List[Foo]]("[1, 2, 3]")(deriveDecoder[List[Foo]])
val res2: Either[io.circe.Error,List[Foo]] = Left(DecodingFailure(CNil, List()))

This compiles because Scala's List is an algebraic data type that has a generic representation that circe-generic can create an instance for. The decoding fails for this input, though, since this representation doesn't result in the encoding we expect. We can derive the corresponding encoder to see what this encoding looks like:

scala> import io.circe.Encoder, io.circe.generic.semiauto.deriveEncoder
import io.circe.Encoder
import io.circe.generic.semiauto.deriveEncoder

scala> implicit val encodeFoo: Encoder[Foo] = Encoder[Int].contramap(_.i)
val encodeFoo: io.circe.Encoder[Foo] = io.circe.Encoder$$anon$1@2717857a

scala> deriveEncoder[List[Foo]].apply(List(Foo(1), Foo(2)))
val res3: io.circe.Json =
{
  "::" : [
    1,
    2
  ]
}

So we're actually seeing the :: case class for List, which is basically never what we want.

If you need to provide a Decoder[List[Foo]] explicitly, the solution is to use either the Decoder.apply "summoner" method, or to call Decoder.decodeList explicitly:

scala> Decoder[List[Foo]]
val res4: io.circe.Decoder[List[Foo]] = io.circe.Decoder$$anon$44@5d40f590

scala> Decoder.decodeList[Foo]
val res5: io.circe.Decoder[List[Foo]] = io.circe.Decoder$$anon$44@2f936a01

scala> Decoder.decodeList(decodeFoo)
val res6: io.circe.Decoder[List[Foo]] = io.circe.Decoder$$anon$44@7f525e05

These all provide exactly the same instance, and which you should choose is a matter of taste.

As a footnote, I've thought about special-casing List in circe-generic so that deriveDecoder[List[X]] doesn't compile, since it's approximately never what you want (but seems like it might be, especially because of the confusing way we talk about instance derivation). I typically don't like the idea of having special cases like that, but I think in this case it might be the right thing to do, since this question comes up a lot.

Upvotes: 3

Related Questions