Why does C++11's lambda require "mutable" keyword for capture-by-value, by default?

Question

Short example:

#include <iostream>

int main()
{
    int n;
    [&](){n = 10;}();             // OK
    [=]() mutable {n = 20;}();    // OK
    // [=](){n = 10;}();          // Error: a by-value capture cannot be modified in a non-mutable lambda
    std::cout << n << "\n";       // "10"
}

The question: Why do we need the mutable keyword? It's quite different from traditional parameter passing to named functions. What's the rationale behind?

I was under the impression that the whole point of capture-by-value is to allow the user to change the temporary -- otherwise I'm almost always better off using capture-by-reference, aren't I?

Any enlightenments?

(I'm using MSVC2010 by the way. AFAIK this should be standard)

Answer 1

This really mixes up two questions into one, and separating them makes it easier to provide answers:

1. Why is a lambda capturing by value that modifies a captured variable ill-formed?
2. Why are values captured by lambdas const by default?

Question #1 has a simple factual answer: Values captured by lambdas are const by default. In these cases:

int n = 0;
[=](){n = 10;}();
[n](){n = 10;}();
[val = n](){val = 10;}();

n or val are basically "members" of the lambdas of type const int. You cannot modify a member of a class or struct that has type const int, because it is const.

Question #2 is a question about language design, and the answer can be complicated or opinionated. Many possible answers have been given in the other answers here already, but many of them do not answer question #1, so I think they are not helpful for visitors who are only interested in how lambda capture works, not discuss language design principles and reasoning.

Answer 2

You might see the difference, if you check 3 different use cases of lambda:

• Capturing an argument by value
• Capturing an argument by value with 'mutable' keyword
• Capturing an argument by reference

case 1: When you capture an argument by value, a few things happen:

• You are not allowed to modify the argument inside the lambda
• The value of the argument remains the same, whenever the lambda is called, not matter what will be the argument value at the time the lambda is called.

so for example:

{
    int x = 100;
    auto lambda1 = [x](){
      // x += 2;  // compile time error. not allowed
                  // to modify an argument that is captured by value
      return x * 2;
    };
    cout << lambda1() << endl;  // 100 * 2 = 200
    cout << "x: " << x << endl; // 100

    x = 300;
    cout << lambda1() << endl;   // in the lambda, x remain 100. 100 * 2 = 200
    cout << "x: " << x << endl;  // 300

}

Output: 
200
x: 100
200
x: 300

case 2: Here, when you capture an argument by value and use the 'mutable' keyword, similar to the first case, you create a "copy" of this argument. This "copy" lives in the "world" of the lambda, but now, you can actually modify the argument within the lambda-world, so its value is changed, and saved and it can be referred to, in the future calls of this lambda. Again, the outside "life" of the argument might be totally different (value wise):

{
    int x = 100;
    auto lambda2 = [x]() mutable {
      x += 2;  // when capture by value, modify the argument is
               // allowed when mutable is used.
      return x;
    };
    cout << lambda2() << endl;  // 100 + 2 = 102
    cout << "x: " << x << endl; // in the outside world - x remains 100
    x = 200;
    cout << lambda2() << endl;  // 104, as the 102 is saved in the lambda world.
    cout << "x: " << x << endl; // 200
}

Output:
102
x: 100
104
x: 200

case 3: This is the easiest case, as no more 2 lives of x. Now there is only one value for x and it's shared between the outside world and the lambda world.

{
    int x = 100;
    auto lambda3 = [&x]() mutable {
        x += 10;  // modify the argument, is allowed when mutable is used.
        return x;
    };
    cout << lambda3() << endl;  // 110
    cout << "x: " << x << endl; // 110
    x = 400;
    cout << lambda3() << endl;  // 410.
    cout << "x: " << x << endl; // 410
}

Output: 
110
x: 110
410
x: 410

Answer 3

I also was wondering about it and the simplest explanation why [=] requires explicit mutable is in this example:

int main()
{
    int x {1};
    auto lbd = [=]() mutable { return x += 5; };
    printf("call1:%d\n", lbd());
    printf("call2:%d\n", lbd());
    return 0;
}

Output:

call1:6
call2:11

By words:

You can see that the x value is different at the second call (1 for the call1 and 6 for the call2).

1. A lambda object keeps a captured variable by value (has its own copy) in case of [=].
2. The lambda can be called several times.

And in general case we have to have the same value of the captured variable to have the same predictable behavior of the lambda based on the known captured value, not updated during the lambda work. That's why the default behavior assumed const (to predict changes of the lambda object members) and when a user is aware of consequences he takes this responsibility on himself with mutable.

Same with capturing by value. For my example:

auto lbd = [x]() mutable { return x += 5; };

Answer 4

You have to understand what capture means! it's capturing not argument passing! let's look at some code samples:

int main()
{
    using namespace std;
    int x = 5;
    int y;
    auto lamb = [x]() {return x + 5; };

    y= lamb();
    cout << y<<","<< x << endl; //outputs 10,5
    x = 20;
    y = lamb();
    cout << y << "," << x << endl; //output 10,20

}

As you can see even though x has been changed to 20 the lambda is still returning 10 ( x is still 5 inside the lambda) Changing x inside the lambda means changing the lambda itself at each call (the lambda is mutating at each call). To enforce correctness the standard introduced the mutable keyword. By specifying a lambda as mutable you are saying that each call to the lambda could cause a change in the lambda itself. Let see another example:

int main()
{
    using namespace std;
    int x = 5;
    int y;
    auto lamb = [x]() mutable {return x++ + 5; };

    y= lamb();
    cout << y<<","<< x << endl; //outputs 10,5
    x = 20;
    y = lamb();
    cout << y << "," << x << endl; //outputs 11,20

}

The above example shows that by making the lambda mutable, changing x inside the lambda "mutates" the lambda at each call with a new value of x that has no thing to do with the actual value of x in the main function

Answer 5

To extend Puppy's answer, lambda functions are intended to be pure functions. That means every call given a unique input set always returns the same output. Let's define input as the set of all arguments plus all captured variables when the lambda is called.

In pure functions output solely depends on input and not on some internal state. Therefore any lambda function, if pure, does not need to change its state and is therefore immutable.

When a lambda captures by reference, writing on captured variables is a strain on the concept of pure function, because all a pure function should do is return an output, though the lambda does not certainly mutate because the writing happens to external variables. Even in this case a correct usage implies that if the lambda is called with the same input again, the output will be the same everytime, despite these side effects on by-ref variables. Such side effects are just ways to return some additional input (e.g. update a counter) and could be reformulated into a pure function, for example returning a tuple instead of a single value.

Answer 6

I was under the impression that the whole point of capture-by-value is to allow the user to change the temporary -- otherwise I'm almost always better off using capture-by-reference, aren't I?

n is not a temporary. n is a member of the lambda-function-object that you create with the lambda expression. The default expectation is that calling your lambda does not modify its state, therefore it is const to prevent you from accidentally modifying n.

Answer 7

FWIW, Herb Sutter, a well-known member of the C++ standardization committee, provides a different answer to that question in Lambda Correctness and Usability Issues:

Consider this straw man example, where the programmer captures a local variable by value and tries to modify the captured value (which is a member variable of the lambda object):

int val = 0;
auto x = [=](item e)            // look ma, [=] means explicit copy
            { use(e,++val); };  // error: count is const, need ‘mutable’
auto y = [val](item e)          // darnit, I really can’t get more explicit
            { use(e,++val); };  // same error: count is const, need ‘mutable’

This feature appears to have been added out of a concern that the user might not realize he got a copy, and in particular that since lambdas are copyable he might be changing a different lambda’s copy.

His paper is about why this should be changed in C++14. It is short, well written, worth reading if you want to know "what's on [committee member] minds" with regards to this particular feature.

Answer 8

There is now a proposal to alleviate the need for mutable in lambda declarations: n3424

Answer 9

Your code is almost equivalent to this:

#include <iostream>

class unnamed1
{
    int& n;
public:
    unnamed1(int& N) : n(N) {}

    /* OK. Your this is const but you don't modify the "n" reference,
    but the value pointed by it. You wouldn't be able to modify a reference
    anyway even if your operator() was mutable. When you assign a reference
    it will always point to the same var.
    */
    void operator()() const {n = 10;}
};

class unnamed2
{
    int n;
public:
    unnamed2(int N) : n(N) {}

    /* OK. Your this pointer is not const (since your operator() is "mutable" instead of const).
    So you can modify the "n" member. */
    void operator()() {n = 20;}
};

class unnamed3
{
    int n;
public:
    unnamed3(int N) : n(N) {}

    /* BAD. Your this is const so you can't modify the "n" member. */
    void operator()() const {n = 10;}
};

int main()
{
    int n;
    unnamed1 u1(n); u1();    // OK
    unnamed2 u2(n); u2();    // OK
    //unnamed3 u3(n); u3();  // Error
    std::cout << n << "\n";  // "10"
}

So you could think of lambdas as generating a class with operator() that defaults to const unless you say that it is mutable.

You can also think of all the variables captured inside [] (explicitly or implicitly) as members of that class: copies of the objects for [=] or references to the objects for [&]. They are initialized when you declare your lambda as if there was a hidden constructor.

Answer 10

It requires mutable because by default, a function object should produce the same result every time it's called. This is the difference between an object orientated function and a function using a global variable, effectively.

Answer 11

You need to think what is the closure type of your Lambda function. Every time you declare a Lambda expression, the compiler creates a closure type, which is nothing less than an unnamed class declaration with attributes (environment where the Lambda expression where declared) and the function call ::operator() implemented. When you capture a variable using copy-by-value, the compiler will create a new const attribute in the closure type, so you can't change it inside the Lambda expression because it is a "read-only" attribute, that's the reason they call it a "closure", because in some way, you are closing your Lambda expression by copying the variables from upper scope into the Lambda scope. When you use the keyword mutable, the captured entity will became a non-const attribute of your closure type. This is what causes the changes done in the mutable variable captured by value, to not be propagated to upper scope, but keep inside the stateful Lambda. Always try to imagine the resulting closure type of your Lambda expression, that helped me a lot, and I hope it can help you too.

Answer 12

See this draft, under 5.1.2 [expr.prim.lambda], subclause 5:

The closure type for a lambda-expression has a public inline function call operator (13.5.4) whose parameters and return type are described by the lambda-expression’s parameter-declaration-clause and trailingreturn- type respectively. This function call operator is declared const (9.3.1) if and only if the lambdaexpression’s parameter-declaration-clause is not followed by mutable.

Edit on litb's comment: Maybe they thought of capture-by-value so that outside changes to the variables aren't reflected inside the lambda? References work both ways, so that's my explanation. Don't know if it's any good though.

Edit on kizzx2's comment: The most times when a lambda is to be used is as a functor for algorithms. The default constness lets it be used in a constant environment, just like normal const-qualified functions can be used there, but non-const-qualified ones can't. Maybe they just thought to make it more intuitive for those cases, who know what goes on in their mind. :)

Answer 13

I was under the impression that the whole point of capture-by-value is to allow the user to change the temporary -- otherwise I'm almost always better off using capture-by-reference, aren't I?

The question is, is it "almost"? A frequent use-case appears to be to return or pass lambdas:

void registerCallback(std::function<void()> f) { /* ... */ }

void doSomething() {
  std::string name = receiveName();
  registerCallback([name]{ /* do something with name */ });
}

I think that mutable isn't a case of "almost". I consider "capture-by-value" like "allow me to use its value after the captured entity dies" rather than "allow me to change a copy of it". But perhaps this can be argued.