How to extract label from Shapeless record for generic type class derivation

Question

I'm writing a generic type class (codec for DynamoDB) deriving code with Shapeless. I have a version that works without using the case classes' field names, based purely on idea that order of classes' fields match the order of attributes in DynamoDB response. It uses Generic and usual deriveHNil, deriveHCons approach, described, for example, here: https://meta.plasm.us/posts/2015/11/08/type-classes-and-generic-derivation/.

Now I want a version that uses the field names to find the relevant DynamoDB attributes. My current idea is to mostly reuse the approach from the previous (order based) version, and additionally make compiler provide the field names via LabelledGeneric and shapeless.ops.record.Keys. However I'm stuck on how to correctly use the Keys functionality.

The idea is the following: function hconsDecoder should do 2 things at a time: deconstruct HList to run decode operation on its head + tail, and also extract the label from the aforementioned head. LabelledGeneric should provide the HList with the labels on the fields, so that H type parameter in hconsDecoder will be an entry in the record, containing the relevant information. But because Keys work only on a HList, I create a singleton H :: HNil to run the Keys on.

Here's the part of code I have:

trait FieldDecoder[A] {
  def decode(a: AttributeValue): Option[A]
}

trait RecordDecoder[A] {
  def decode(s: Seq[Attribute]): Option[A]
}

object RecordDecoderInstances {

  implicit val hnilDecoder = new RecordDecoder[HNil] {
    override def decode(s: Seq[Attribute]): Option[HNil] = {
      Some(HNil)
    }
  }

  object toName extends Poly1 {
    implicit def keyToName[A] = at[Symbol with A](_.name)
  }

  implicit def hconsDecoder[H: FieldDecoder, T <: HList: RecordDecoder](
      implicit kk: Keys[H :: HNil]#Out,
      m: Mapper[toName.type, Keys[H :: HNil]#Out]) =
    new RecordDecoder[H :: T] {
      override def decode(s: Seq[Attribute]): Option[H :: T] = {

        val attrName = (kk map toName).head.asInstanceOf[String] // compile error here

        for {
          h <- implicitly[FieldDecoder[H]]
            .decode(s.filter(_.name == attrName).head.value)
          t <- implicitly[RecordDecoder[T]]
            .decode(s.filterNot(_.name == attrName))
        } yield h :: t

      }
    }
}

Given this code, the compiler error is the following: could not find implicit value for parameter c: shapeless.ops.hlist.IsHCons[m.Out]. I've tried different versions of the same, always facing some variation of implicit not found error. Bottom line is, Keys doesn't work with H :: HNil construct for some reason.

This is my first serious attempt at Shapeless, and I don't know if I'm going the right way. I'd appreciate both feedback on this particular error and on my approach in general.

Answer 1

I was scavenging Github for the inspiration, and found some in Frameless project. It seems that using the Witness together with LabelledGeneric gives you direct access to the field names. I've came up with the following version that works:

trait Decoder[A] {
  def decode(s: Seq[Attribute]): Option[A]
}

object Decoder {
  implicit val hnilDecoder = new Decoder[HNil] {
    override def decode(s: Seq[Attribute]): Option[HNil] = {
      Some(HNil)
    }
  }
  implicit def keyedHconsDecoder[K <: Symbol, H, T <: HList](
      implicit key: Witness.Aux[K],
      head: FieldDecoder[H],
      tail: Decoder[T]
  ): Decoder[FieldType[K, H] :: T] =
    new Decoder[FieldType[K, H] :: T] {
      def decode(s: Seq[Attribute]) = {
        val fieldName = key.value.name
        for {
          head <- head.decode(s, fieldName)
          tail <- tail.decode(s)
        } yield labelled.field[K](head) :: tail
      }
    }
  implicit def caseClassDecoder[A, R <: HList](
      implicit gen: LabelledGeneric.Aux[A, R],
      reprDecoder: Lazy[Decoder[R]],
      ct: ClassTag[A]): Decoder[A] =
    new Decoder[A] {
      override def decode(s: Seq[Attribute]): Option[A] = {
        println(s"record decode case class ${ct.runtimeClass.getSimpleName}")
        reprDecoder.value.decode(s).map(gen.from)
      }
    }
  def apply[A](s: Seq[Attribute])(implicit decoder: Decoder[A],
                                  ct: ClassTag[A]): Option[A] = {
    println(s"start decoding for ${ct.runtimeClass}")
    decoder.decode(s)
  }
}

(omitting the FieldDecoder for brevity)

One also needs to adjust the return type in HCons decoder (keyedHconsDecoder) from Decoder[H :: T] to Decoder[[FieldType[K, H] :: T], because we're dealing with LabelledGeneric here.

Answer 2

I was facing the same problem before but didn't find a straightforward approach for aligning the Keys HList with the Generic HList recursively. I hope someone will post a better solution.

A simple solution would be to align the input sequence with the keys before the recursive processing. I have separated the seq processing from the processing of the generic HList representation for clarity.

case class AttributeValue(value: String)
case class Attribute(name: String, value: AttributeValue)

trait FieldDecoder[T] {
  def decode(a: AttributeValue): Option[T]
}

HList decoder:

trait HListDecoder[A <: HList] {
  def decode(s: Seq[Attribute]): Option[A]
}

object HListDecoder {

  implicit val hnilDecoder = new HListDecoder[HNil] {
    override def decode(s: Seq[Attribute]): Option[HNil] = {
      Some(HNil)
    }
  }

  implicit def hconsDecoder[H, T <: HList, LR <: HList](
    implicit
      fieldDecoder: FieldDecoder[H],
      tailDecoder: HListDecoder[T]) =
    new HListDecoder[H :: T] {
      override def decode(s: Seq[Attribute]): Option[H :: T] = {
        for {
          h <- fieldDecoder.decode(s.head.value)
          t <- tailDecoder.decode(s.tail)
        } yield h :: t

      }
    }
}

Case class decoder:

trait RecordDecoder[A] {
  def decode(s: Seq[Attribute]): Option[A]
}

object RecordDecoder {

  object toName extends Poly1 {
    implicit def keyToName[A] = at[Symbol with A](_.name)
  }

  def sortByKeys(s: Seq[Attribute], keys: Seq[String]): Seq[Attribute] =
    keys.flatMap(key => s.filter(_.name == key))

  implicit def recordDecoder[A, R <: HList, LR <: HList, K <: HList, KL <: HList](
    implicit
      gen: Generic.Aux[A, R],
      lgen: LabelledGeneric.Aux[A, LR],
      kk: Keys.Aux[LR, K],
      m: Mapper.Aux[toName.type, K, KL],
      toSeq: ToTraversable.Aux[KL, Seq, String],
      genDecoder: HListDecoder[R]): RecordDecoder[A] =
    new RecordDecoder[A] {
      def decode(s: Seq[Attribute]) = {
        val keys = kk.apply.map(toName).to[Seq]
        val attrs = sortByKeys(s, keys)
        genDecoder.decode(attrs).map(gen.from _)
      }
    }

  def apply[A](s: Seq[Attribute])(implicit decoder: RecordDecoder[A]) =
    decoder.decode(s)

}

Test case:

implicit val stringDecoder = new FieldDecoder[String] {
  override def decode(a: AttributeValue): Option[String] = Some(a.value)
}

implicit val intDecoder = new FieldDecoder[Int] {
  override def decode(a: AttributeValue): Option[Int] = Some(a.value.toInt)
}

val attrs = Seq(
  Attribute("a", new AttributeValue("a")),
  Attribute("c", new AttributeValue("2")),
  Attribute("b", new AttributeValue("b")))

case class Test(b: String, a: String, c: Int)

println(RecordDecoder[Test](attrs))

// Some(Test(b,a,2))