CODEWITHSUNDEEP
scalaapache-spark
Guforu
Guforu

Reputation: 4023

Class ID in Scala

The case class Rating is a part of ALS.scala I can not find the source code for ID class (or object). Can somebody share the link for this class? It should be open source. How I understand, the class ID is not a part of ALS.scala, anyway I can not find import statement for it.

Upvotes: 0

Views: 421

Answers (1)

eliasah
eliasah

Reputation: 40380

That's because that implementation of ALS supports generic ID types, specialized for Int and Long. So ID isn't a case class. This is what the following line means :

case class Rating[@specialized(Int, Long) ID](user: ID, item: ID, rating: Float)

Explanation :

Generics in Java and the JVM, and consequently also in Scala, are implemented by using type erasure. That means that if you have an instance of a generic class, for example List[String], then the compiler will throw away the information about the type argument, so that at runtime it will look like List[Object].

In Java, primitive types are treated very differently from reference types. One of the things that you cannot do in Java is use primitive types to fill in type parameters – for example, you cannot make a List<int>. If you want to create a list that holds integers, you’ll have to use the wrapper class Integer. A drawback of this approach is that you need to box each of your primitive ints into an Integer object that takes up a lot more memory. I’ve done some tests and found that on my JVM a double only takes up 8 bytes, but a Double object takes up 24 bytes. Also, the boxing and unboxing takes up processor time, which can become a serious bottleneck if you’re dealing with large collections of numbers.

In Scala, the distinction between primitive types and reference types is far less great than in Java, and you can use types like Int directly for type arguments. But a List[Int] is still converted to a List[Object] at runtime because of type erasure and since primitive types are not subclasses of Object on the JVM, Scala will still box the Ints to objects of some wrapper class – which makes a List[Int] in Scala just as inefficient as a List<Integer> in Java.

The @specialized annotation and compiler support are meant to get rid of this inefficiency.

More on Specializing for primitive types here.

Upvotes: 2

Related Questions