Reading mutable references has different lifetime semantics than immutable references

Question

Consider the following code, where a reference to a root type R is wrapped. Also stored is some type N(avigate), which knows how to dereference R for T.

use std::ops::Deref;

struct Wrapper<'r, R, N, T>
where
    N: Fn(&'r R) -> &T,
    T: 'static,
{
    r: &'r R,
    n: N,
}

impl<'r, R, N, T> Deref for Wrapper<'r, R, N, T>
where
    N: Fn(&'r R) -> &T,
    T: 'static,
{
    type Target = T;

    fn deref(&self) -> &T {
        let r: &'r R = self.r;
        let t: &'r T = (self.n)(r);
        t
    }
}

Now, if we change our reference type r: &'r R, to be mutable r: &'r mut R, it no longer works:

use std::ops::Deref;

struct Wrapper<'r, R, N, T>
where
    N: Fn(&'r R) -> &T,
    T: 'static,
{
    r: &'r mut R,
    n: N,
}

impl<'r, R, N, T> Deref for Wrapper<'r, R, N, T>
where
    N: Fn(&'r R) -> &T,
    T: 'static,
{
    type Target = T;

    fn deref(&self) -> &T {
        let r: &'r R = self.r;
        let t: &'r T = (self.n)(r);
        t
    }
}

Error:

error[E0312]: lifetime of reference outlives lifetime of borrowed content...
  --> src/lib.rs:21:24
   |
21 |         let r: &'r R = self.r;
   |                        ^^^^^^
   |
note: ...the reference is valid for the lifetime 'r as defined on the impl at 13:6...
  --> src/lib.rs:13:6
   |
13 | impl<'r, R, N, T> Deref for Wrapper<'r, R, N, T>
   |      ^^
note: ...but the borrowed content is only valid for the anonymous lifetime #1 defined on the method body at 20:5
  --> src/lib.rs:20:5
   |
20 | /     fn deref(&self) -> &T {
21 | |         let r: &'r R = self.r;
22 | |         let t: &'r T = (self.n)(r);
23 | |         t
24 | |     }
   | |_____^

We get a better error message with nll:

error: lifetime may not live long enough
  --> src/lib.rs:21:16
   |
13 | impl<'r, R, N, T> Deref for Wrapper<'r, R, N, T>
   |      -- lifetime `'r` defined here
...
20 |     fn deref(&self) -> &T {
   |              - let's call the lifetime of this reference `'1`
21 |         let r: &'r R = self.r;
   |                ^^^^^ type annotation requires that `'1` must outlive `'r

I've annotated the lifetimes in deref to make sure i'm on the same track as the compiler about the lifetimes. The nll message is particulary interesting, because it says that it requires &self to outlive 'r.

But that doesn't make sense to me, since if we annotate the lifetimes on deref, it should look like this:

fn deref<'1>(&'1 self) -> &'1 T;

And rather require that 'r: '1, which is implicitly given by Wrapper<'r, ...>

This intuition seems to hold in the first example, but not in the second with the immutable reference.

So two questions unfold for me:

1. Why does it make a difference if self.r is immutable or not? I can't access r mutably anyway, since &self is immutable.
2. Is 1. a fundamental restriction, or can the code be annotated in a way to tell rustc what i want to do?

Answer 1

Trait Types are invariant over their generic parameters.

Consider this example:

struct Test<'a, F: Fn(&'a i32)> {
    i: &'a i32,
    f: F,
}

fn main() {
    let i = 1i32;
    let t = Test { i: &i, f: |&_| {} };

    {
        let j = 2i32;
        (t.f)(&j);
    }

    println!("{:?}", t.i);
}

This will give the error:

error[E0597]: `j` does not live long enough
  --> src/main.rs:12:15
   |
12 |         (t.f)(&j);
   |               ^^ borrowed value does not live long enough
13 |     }
   |     - `j` dropped here while still borrowed
14 | 
15 |     println!("{:?}", t.i);
   |                      --- borrow later used here

As you can see, the type Test<'a ... is not unified to a shorter lifetime with that of j because Test contains a trait impl type N (static dispatch). As a result, it will be invariant over 'a, hence 'a cannot be shortened. But j does not live for 'a, hence the error.

Moving to your question, let's have a look at a minimal version of your code:

struct Wrapper<'r, R, N>
where
    N: Fn(&'r R),
{
    r: &'r mut R,
    n: N,
}

impl<'r, R, N> Wrapper<'r, R, N>
where
    N: Fn(&'r R),
{
    fn myderef(&self) {
        (self.n)(self.r)
    }
}

This would give the same error:

error[E0312]: lifetime of reference outlives lifetime of borrowed content...
  --> src/lib.rs:14:18
   |
14 |         (self.n)(self.r)
   |                  ^^^^^^
   |
note: ...the reference is valid for the lifetime 'r as defined on the impl at 9:6...
  --> src/lib.rs:9:6
   |
9  | impl<'r, R, N> Wrapper<'r, R, N>
   |      ^^
note: ...but the borrowed content is only valid for the anonymous lifetime #1 defined on the method body at 13:5
  --> src/lib.rs:13:5
   |
13 | /     fn myderef(&self) {
14 | |         (self.n)(self.r)
15 | |     }
   | |_____^

What's exactly happening here? &self with lifetimes will be of type &'shorter_lifetime Wrapper<'r, R, N> and not &'shorter_lifetime Wrapper<'shorter_lifetime, R, N>. 'r will not be shortened to 'shorter_lifetime as Wrapper will be invariant over it's generic lifetime parameter 'r because of N.

Now that we know what exactly the argument type &self is, let's see what happens inside the body of myderef(). The trait type N (static dispatch) is invoked with self.r. But self.r is a mutable reference, that gets re-borrowed when passed to (self.r)(). So now you have a mutable reference that is behind another reference (self is a reference), that needs to live for 'r (N needs it's input argument to be with 'r lifetime as per the definition), as a result &self too needs to live for 'r. But &self's lifetime is 'shorter_lifetime, hence the error.

To put it another way, if you have &'a & 'b mut T (no subtyping relation between 'a and 'b) as an input argument to a function and the compiler allows you to reborrow the inner reference and return it, then there is a violation of the borrowing rules, since &mut T is already behind a reference. The outer reference "owns" the inner reference, mainly because the inner reference is mutable and the function needs a guarantee that the outer reference remains for at-least as long as the inner (mutable) reference is reborrowed, otherwise after the function call there would be more than one owner of the mutable reference.

As an example, the following code will not compile:

fn test<'a, 'b> (i:&'a &'b mut i32) -> &'b i32 {
    &**i
}

But this one will:

fn test<'a:'b, 'b> (i:&'a &'b mut i32) -> &'b i32 {
    &**i
}

as there's a guarantee that 'a lives for at-least as long as 'b.

If the inner reference is immutable, then the former will also compile as you can have multiple immutable references. There is no notion of outer reference "owning" the inner reference.

To make the minimal version compile, we will have to tell the compiler that &self also lives for 'r. Either that or remove the hard constraint of 'r on the N's input argument (lifetime elision).

In your example, deref() won't allow you to specify a lifetime on &self, as per Deref's definition. If you remove the hard constraint of 'r on N's input argument, it will compile