Scala by-name parameter with default null throws NullPointerException

Question

The following snippet throws NullPointerException. Is it expected and normal behavior of Scala?

object ATest extends App {
    def getX[T <: X](constr: ⇒ T = null.asInstanceOf[T]): Unit = {
        constr
    }
    getX()
}
class X

Generated (decompied) Java code from snippet:

public final class ATest {
public static void main(String[] arrstring) {
    ATest$.MODULE$.main(arrstring);
}
public static void delayedInit(Function0 function0) {
    ATest$.MODULE$.delayedInit(function0);
}
public static String[] args() {
    return ATest$.MODULE$.args();
}
public static void scala$App$_setter_$executionStart_$eq(long l) {
    ATest$.MODULE$.scala$App$_setter_$executionStart_$eq(l);
}
public static long executionStart() {
    return ATest$.MODULE$.executionStart();
}
public static void delayedEndpoint$test$ATest$1() {
    ATest$.MODULE$.delayedEndpoint$test$ATest$1();
}
public static  T getX$default$1() {
    return ATest$.MODULE$.getX$default$1();
}
public static  void getX(Function0 function0) {
    ATest$.MODULE$.getX(function0);
}
}


public final class ATest$ implements App {
public static final ATest$ MODULE$;
private final long executionStart;
private String[] scala$App$$_args;
private final ListBuffer> scala$App$$initCode;

public static {
    new test.ATest$();
}
public long executionStart() {
    return this.executionStart;
}
public String[] scala$App$$_args() {
    return this.scala$App$$_args;
}
public void scala$App$$_args_$eq(String[] x$1) {
    this.scala$App$$_args = x$1;
}
public ListBuffer> scala$App$$initCode() {
    return this.scala$App$$initCode;
}
public void scala$App$_setter_$executionStart_$eq(long x$1) {
    this.executionStart = x$1;
}
public void scala$App$_setter_$scala$App$$initCode_$eq(ListBuffer x$1) {
    this.scala$App$$initCode = x$1;
}
public String[] args() {
    return App.class.args((App)this);
}
public void delayedInit(Function0 body) {
    App.class.delayedInit((App)this, body);
}
public void main(String[] args) {
    App.class.main((App)this, (String[])args);
}
public  void getX(Function0 constr) {
    constr.apply();
}
public  T getX$default$1() {
    return null;
}
public final void delayedEndpoint$test$ATest$1() {
    this.getX((Function0)new scala.Serializable(){
        public static final long serialVersionUID = 0;

        public final Nothing. apply() {
            return (Nothing.)ATest$.MODULE$.getX$default$1();
        }
    });
}
private ATest$() {
    MODULE$ = this;
    App.class.$init$((App)this);
    this.delayedInit((Function0)new ATest.delayedInit$body(this));
}
}

public final class ATest$.anonfun extends AbstractFunction0implements Serializable {
public final Nothing. apply() {
        return (Nothing.)ATest$.MODULE$.getX$default$1();
        }
}

And finally action part:

public  void getX(Function0 constr) {
    constr.apply();
}
public  T getX$default$1() {
    return null;
}
public final void delayedEndpoint$test$ATest$1() {
    this.getX((Function0)new scala.Serializable(){
        public final Nothing. apply() {
            return (Nothing.)ATest$.MODULE$.getX$default$1();
        }
    });
}

That is: call to getX passes new anon Function0 which apply() just calls getX$default$1() that is null. So i can not see any point where NPE can be thrown.

EDIT: The unresolved issue is found: https://issues.scala-lang.org/browse/SI-8097

EDIT: Expression null.asInstanceOf[T] generates default value for type T. In case when Scala infers resulting type T as Nothing we come to runtime expression

null.asInstanceOf[Nothing]

that obviously throws Exeption as a default for Nothing is Exception.

But than why this snippet throws NPE only at last line?

object ATest extends App {
    def getX[T](x: T = null.asInstanceOf[T]): T = x
    getX[Nothing]() // Ok
    val x = getX() // Ok 
    val y = null
    println("x= "+x) // prints 'x= null'
    println(s"y= $y") // prints 'y= null'
    println(s"x= $x") // throws NPE !?
    println("x==null ? "+(x==null)) // prints 'x= null'
}

And why this snippet throws NPE (it is only different from previous in implicit param)?

object ATest extends App {
    def getX[T](x: T = null.asInstanceOf[T])(implicit s: String = null): T = x
    getX() // throws NPE !?
}

So the situation is still vague. And question is open.

dth · Accepted Answer

So I have to revise my answer a bit.

What triggers the NPE is however clear from the byte code, but not the reverse compiled Java code. Byte code has more features than Java code, the important one being, that you can have two methods that differ only in the return type and do different things.

So lets first look at the stack trace:

at ATest$$anonfun$1.apply(Test.scala:7)
at ATest$.getX(Test.scala:5)
at ATest$.delayedEndpoint$ATest$1(Test.scala:7)
at ATest$delayedInit$body.apply(Test.scala:3)
at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:383)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at ATest$.main(Test.scala:3)
at ATest.main(Test.scala)

So the method where it all goes wrong is ATest$$anonfun$1.apply

Lets look at that:

public final scala.runtime.Nothing$ apply();
Code:
   0: getstatic     #19                 // Field ATest$.MODULE$:LATest$;
   3: invokevirtual #23                 // Method ATest$.getX$default$1:()LX;
   6: checkcast     #25                 // class scala/runtime/Nothing$
   9: areturn

public final java.lang.Object apply();
Code:
   0: aload_0
   1: invokevirtual #30                 // Method apply:()Lscala/runtime/Nothing$;
   4: athrow

The first thing we notice is there are two Methods called apply, so which one is called (the athrow is a hint...) Well, lets look at the method calling it:

public  void getX(scala.Function0);
Code:
   0: aload_1
   1: invokeinterface #62,  1  // InterfaceMethod scala/Function0.apply:()Ljava/lang/Object;
   6: pop
   7: return

So, we are calling the one that returns an Object and has the athrow instruction. So why does that give us a NullPointer exception?

Well, the method does the following: it places this on the stack, then it invokes the other apply method (returning a Nothing$), this method actually returns null as it returns our default argument. And now we have a null on the stack and execute athrow. And athrow throws a NPE instead if it finds a null on the stack.

So this is what happens here.

The next question is, why does it happen?

Well, lets look at what scalac makes of it after typechecking:

object ATest extends AnyRef with App {
    def (): ATest.type = {
      ATest.super.();
      ()
    };
    def getX[T <: X](constr: => T = null.asInstanceOf[T]): Unit = {
      constr;
      ()
    };
     def getX$default$1[T <: X]: T = null.asInstanceOf[T];
    ATest.this.getX[Nothing](ATest.this.getX$default$1[Nothing])
  }

And what it does in the case without the asInstanceOf:

object ATest extends AnyRef with App {
    def (): ATest.type = {
      ATest.super.();
      ()
    };
    def getX[T <: X](constr: => T = null): Unit = {
      constr;
      ()
    };
     def getX$default$1[T <: X]: Null = null;
    ATest.this.getX[Null](ATest.this.getX$default$1[Nothing])
  }

Well, somehow the information, that the default parameter is Null is lost in the first case.

In the second case we get this byte code for the critical method:

public final java.lang.Object apply();
Code:
   0: aload_0
   1: invokevirtual #27                 // Method apply:()Lscala/runtime/Null$;
   4: pop
   5: aconst_null
   6: areturn

So here, the compiler knows, that the argument is null, and generates code to box null using the class Null$.

What should happen?

Well, not a null pointer exception, for sure. But why does the compiler generate that athrow in the first place? Probably because of the asInstanceOf[T] which becomes an asInstanceOf[Nothing] which should throw an exception if invoked on a null.

Let's quickly try, what happens if we do that in repl:

"".asInstanceOf[Nothing]
java.lang.ClassCastException: java.lang.String cannot be cast to scala.runtime.Nothing$

So far so good, and this:

null.asInstanceOf[Nothing]
java.lang.NullPointerException

Well, maybe I should have started with this... it seems, that code generation for asInstanceOf has some bug and throws the wrong exception.

Why a lower bound :> Null fixes the problem, is also clear: the inferred type is no longer Nothing but Null and the instanceOf is fine.

So the more interesting problem is why the type checker fails on your complex example that you have removed now.

Complex example

class X
object ATest extends App {
  def getX[T<:X](clas: Class[T], constr: ⇒ T = null): T ={
    val x = constr
    if (x == null) clas.newInstance() else x
  }
  val clas: Class[_ <: X] = classOf[X]
  getX(clas) // Ooops: type mismatch..
}

Well, what does the type checker say:

def getX[T <: X](clas: Class[T], constr: => T = null): T = {
  val x: T = constr;
    if (x.==(null))
      clas.newInstance()
    else
      x
  };
   def getX$default$2[T <: X]: Null = null;
  private[this] val clas: Class[_ <: X] = classOf[X];
    def clas: Class[_ <: X] = ATest.this.clas;
  ATest.this.getX[T](, ATest.this.getX$default$2)
}

Somehow he cant infer a type for T, but he should infer Null, because there are only classes for reference types. Interestingly, the compiler does not know that. It seems to directly use the definition from Class from Java and the type parameter there has no lower bound (because Java has no type Null), so the lower bound is Nothing. This also tells us, how to fix it:

val clas: Class[_ >: Null <: X] = classOf[X]
getX(clas)

This finally works. So you can do exactly, what you wanted to do in the first place. You just have to tell the compiler, that you are not interested in classes for types that cannot be nulled.

I think I still prefer the version with Option though:

def getX[T <: X](clas: Class[T], constr: ⇒ Option[T] = None): T = {
  val x = constr
   x match {
    case None => clas.newInstance()
    case Some(x) => x
  }
}
val clas: Class[_ <: X] = classOf[X]
getX(clas)

Its now also clear, why this works: None is a Option[Nothing], so this code can handle a Class[Nothing] just fine.

Scala by-name parameter with default null throws NullPointerException

Answers (1)

The next question is, why does it happen?

What should happen?

Complex example

Related Questions