CODEWITHSUNDEEP
scala
Guille
Guille

Reputation: 2320

Generic with Scala. No ClassTag available for T

I'm trying to use generics in Scala but in some case it doesn't work to me.

I have define a class as:

trait Generic[T <: Product] extends Serializable {

   def start(ssc: StreamingContext, conf: Conf) = {
    val dstream = createDirectStream(...)
    val rdd = dstream.map(x => avroToObject(x.value()))
    execute(rdd)
   }

   def execute(dstreamAvro: DStream[T]): Unit
   def avroToObject(bytes: Array[Byte]): T
}

Later, I implement the avroToObject method in a specific class. When I try to compile the code I get an error.

Error:(36, 30) No ClassTag available for T
    val rddAvro = dstream.map(x => avroToObject(x.value()))
Error:(36, 30) not enough arguments for method map: (implicit evidence$2: scala.reflect.ClassTag[T])org.apache.spark.streaming.dstream.DStream[T].
Unspecified value parameter evidence$2.
    val rddAvro = dstream.map(x => avroToObject(x.value()))

What is it happening? How could I fix it?

If I changed the declaration of the class by:

abstract class Generic[T <: Product](implicit c: ClassTag[T]) extends Serializable {..

It works, but I don't understand why, and why it's neccesary that implicit.

Upvotes: 3

Views: 2141

Answers (1)

Yuval Itzchakov
Yuval Itzchakov

Reputation: 149518

What is it happening? How could I fix it?

What is happening is that one of the methods down the execution chain requires an instance of ClassTag[T] to be present. Although we don't have the implementation here, I'm assuming that is the KafkaUtils.createDirectStream method which needs information of type T at run time.

This works:

abstract class Generic[T <: Product](implicit c: ClassTag[T])

Because now that implicit is in scope when you call createDirectStream. That is, the compiler is the one which fills in the relevant class tag at compile time for you, and now createDirectStream has the relevant implicit in scope to use.

Upvotes: 6

Related Questions