Lambdas: local variables need final, instance variables don't

Question

In a lambda, local variables need to be final, but instance variables don't. Why so?

Answer 1

Putting up some concepts for future visitors:

Basically it all boils down to the point that the compiler should be able to deterministically tell that a lambda expression body is not working on a stale copy of the referenced variables.

In the case of local variables, the compiler has no way to be sure that the lambda expression body is not working on a stale copy of the variable unless that variable is final or effectively final, so local variables should either be final or effectively final.

Now, in the case of instance fields, when you access an instance field inside the lambda expression then the compiler will prepend this. to that variable access (if you have not done so explicitly already) and since this is effectively final the compiler is sure that the lambda expression body will always have the latest copy of the variable (please note that multi-threading is out of scope right now for this discussion). So, in the case of instance fields, the compiler can tell that the lambda body has latest copy of instance variable so instance variables need not be final or effectively final. Please refer to the screenshot below from an Oracle slide:

Also, please note that if you are accessing an instance field in a lambda expression which is getting executed in a multi-threaded environment you could potentially run into problems.

Answer 2

In a document of project lambda, State of the Lambda v4, under Section 7. Variable capture, it is mentioned that:

It is our intent to prohibit capture of mutable local variables. The reason is that idioms like this:

int sum = 0;
list.forEach(e -> { sum += e.size(); });

are fundamentally serial; it is quite difficult to write lambda bodies like this that do not have race conditions. Unless we are willing to enforce—preferably at compile time—that such a function cannot escape its capturing thread, this feature may well cause more trouble than it solves.

Another thing to note here is, local variables are passed in the constructor of an inner class when you access them inside your inner class, and this won't work with non-final variable because value of non-final variables can be changed after construction.

While in case of an instance variable, the compiler passes a reference of the object and object reference will be used to access instance variables. So, it is not required in case of instance variables.

PS : It is worth mentioning that anonymous classes can access only final local variables (in Java SE 7), while in Java SE 8 you can access effectively final variables also inside lambda as well as inner classes.

Answer 3

First, there is a key difference in how local and instance variables are implemented behind the scenes. Instance variables are stored in the heap, whereas local variables stored in the stack. If the lambda could access the local variable directly and the lambda was used in a thread, then the thread using the lambda could try to access the variable after the thread that allocated the variable had deallocated it.

In short: to ensure another thread does not override the original value, it is better to provide access to the copy variable rather than the original one.

Answer 4

YES, you can change the member variables of the instance but you CANNOT change the instance itself just like when you handle variables.

Something like this as mentioned:

    class Car {
        public String name;
    }

    public void testLocal() {
        int theLocal = 6;
        Car bmw = new Car();
        bmw.name = "BMW";
        Stream.iterate(0, i -> i + 2).limit(2)
        .forEach(i -> {
//            bmw = new Car(); // LINE - 1;
            bmw.name = "BMW NEW"; // LINE - 2;
            System.out.println("Testing local variables: " + (theLocal + i));

        });
        // have to comment this to ensure it's `effectively final`;
//        theLocal = 2; 
    }

The basic principle to restrict the local variables is about data and computation validity

If the lambda, evaluated by the second thread, were given the ability to mutate local variables. Even the ability to read the value of mutable local variables from a different thread would introduce the necessity for synchronization or the use of volatile in order to avoid reading stale data.

But as we know the principal purpose of the lambdas

Amongst the different reasons for this, the most pressing one for the Java platform is that they make it easier to distribute processing of collections over multiple threads.

Quite unlike local variables, local instance can be mutated, because it's shared globally. We can understand this better via the heap and stack difference:

Whenever an object is created, it’s always stored in the Heap space and stack memory contains the reference to it. Stack memory only contains local primitive variables and reference variables to objects in heap space.

So to sum up, there are two points I think really matter:

1. It's really hard to make the instance effectively final, which might cause lots of senseless burden (just imagine the deep-nested class);

2. the instance itself is already globally shared and lambda is also shareable among threads, so they can work together properly since we know we're handling the mutation and want to pass this mutation around;

Balance point here is clear: if you know what you are doing, you can do it easily but if not then the default restriction will help to avoid insidious bugs.

P.S. If the synchronization required in instance mutation, you can use directly the stream reduction methods or if there is dependency issue in instance mutation, you still can use thenApply or thenCompose in Function while mapping or methods similar.

Answer 5

In Java 8 in Action book, this situation is explained as:

You may be asking yourself why local variables have these restrictions. First, there’s a key difference in how instance and local variables are implemented behind the scenes. Instance variables are stored on the heap, whereas local variables live on the stack. If a lambda could access the local variable directly and the lambda were used in a thread, then the thread using the lambda could try to access the variable after the thread that allocated the variable had deallocated it. Hence, Java implements access to a free local variable as access to a copy of it rather than access to the original variable. This makes no difference if the local variable is assigned to only once—hence the restriction. Second, this restriction also discourages typical imperative programming patterns (which, as we explain in later chapters, prevent easy parallelization) that mutate an outer variable.

Answer 6

The fundamental difference between a field and a local variable is that the local variable is copied when JVM creates a lambda instance. On the other hand, fields can be changed freely, because the changes to them are propagated to the outside class instance as well (their scope is the whole outside class, as Boris pointed out below).

The easiest way of thinking about anonymous classes, closures and labmdas is from the variable scope perspective; imagine a copy constructor added for all local variables you pass to a closure.

Answer 7

Within Lambda expressions you can use effectively final variables from the surrounding scope. Effectively means that it is not mandatory to declare variable final but make sure you do not change its state within the lambda expresssion.

You can also use this within closures and using "this" means the enclosing object but not the lambda itself as closures are anonymous functions and they do not have class associated with them.

So when you use any field (let say private Integer i;)from the enclosing class which is not declared final and not effectively final it will still work as the compiler makes the trick on your behalf and insert "this" (this.i).

private Integer i = 0;
public  void process(){
    Consumer<Integer> c = (i)-> System.out.println(++this.i);
    c.accept(i);
}

Answer 8

Here is a code example, as I didn't expect this either, I expected to be unable to modify anything outside my lambda

 public class LambdaNonFinalExample {
    static boolean odd = false;

    public static void main(String[] args) throws Exception {
       //boolean odd = false; - If declared inside the method then I get the expected "Effectively Final" compile error
       runLambda(() -> odd = true);
       System.out.println("Odd=" + odd);
    }

    public static void runLambda(Callable c) throws Exception {
       c.call();
    }

 }

Output: Odd=true

Answer 9

Because instance variables are always accessed through a field access operation on a reference to some object, i.e. some_expression.instance_variable. Even when you don't explicitly access it through dot notation, like instance_variable, it is implicitly treated as this.instance_variable (or if you're in an inner class accessing an outer class's instance variable, OuterClass.this.instance_variable, which is under the hood this.<hidden reference to outer this>.instance_variable).

Thus an instance variable is never directly accessed, and the real "variable" you're directly accessing is this (which is "effectively final" since it is not assignable), or a variable at the beginning of some other expression.

Answer 10

It seems like you are asking about variables that you can reference from a lambda body.

From the JLS §15.27.2

Any local variable, formal parameter, or exception parameter used but not declared in a lambda expression must either be declared final or be effectively final (§4.12.4), or a compile-time error occurs where the use is attempted.

So you don't need to declare variables as final you just need to make sure that they are "effectively final". This is the same rule as applies to anonymous classes.