Why can't `&(?Sized + Trait)` be cast to `&dyn Trait`?

Question

In the code below it is not possible to obtain a reference to a trait object from a reference to a dynamically-sized type implementing the same trait. Why is this the case? What exactly is the difference between &dyn Trait and &(?Sized + Trait) if I can use both to call trait methods?

A type implementing FooTraitContainerTrait might e.g. have type Contained = dyn FooTrait or type Contained = T where T is a concrete type that implements FooTrait. In both cases it's trivial to obtain a &dyn FooTrait. I can't think of another case where this wouldn't work. Why isn't this possible in the generic case of FooTraitContainerTrait?

trait FooTrait {
    fn foo(&self) -> f64;
}

///

trait FooTraitContainerTrait {
    type Contained: ?Sized + FooTrait;
    fn get_ref(&self) -> &Self::Contained;
}

///

fn foo_dyn(dyn_some_foo: &dyn FooTrait) -> f64 {
    dyn_some_foo.foo()
}

fn foo_generic<T: ?Sized + FooTrait>(some_foo: &T) -> f64 {
    some_foo.foo()
}

///

fn foo_on_container<C: FooTraitContainerTrait>(containing_a_foo: &C) -> f64 {
    let some_foo = containing_a_foo.get_ref();
    // Following line doesn't work:
    //foo_dyn(some_foo)
    // Following line works:
    //some_foo.foo()
    // As does this:
    foo_generic(some_foo)
}

Uncommenting the foo_dyn(some_foo) line results in the compiler error

error[E0277]: the size for values of type `<C as FooTraitContainerTrait>::Contained` cannot be known at compilation time
  --> src/main.rs:27:22
   |
27 |     foo_dyn(contained)
   |             ^^^^^^^^^ doesn't have a size known at compile-time
   |
   = help: the trait `std::marker::Sized` is not implemented for `<C as FooTraitContainerTrait>::Contained`
   = note: to learn more, visit <https://doc.rust-lang.org/book/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait>
   = help: consider adding a `where <C as FooTraitContainerTrait>::Contained: std::marker::Sized` bound
   = note: required for the cast to the object type `dyn FooTrait`

Answer 1

Referenced from this blog, which explains the fat pointer really well.

Thanks trentcl for simplifying the question to:

trait Foo {}

fn make_dyn<T: Foo + ?Sized>(arg: &T) -> &dyn Foo {
    arg
}

This brings to how to cast between different ?Sized?

To answer this, let's first peek the implementation for Unsized type Trait.

trait Bar {
    fn bar_method(&self) {
        println!("this is bar");
    }
}

trait Foo: Bar {
    fn foo_method(&self) {
        println!("this is foo");
    }
}

impl Bar for u8 {}
impl Foo for u8 {}

fn main() {
    let x: u8 = 35;
    let foo: &dyn Foo = &x;
    // can I do
    // let bar: &dyn Bar = foo;
}

So, can you do let bar: &dyn Bar = foo;?

// below is all pseudo code
pub struct TraitObjectFoo {
    data: *mut (),
    vtable_ptr: &VTableFoo,
}

pub struct VTableFoo {
    layout: Layout,
    // destructor
    drop_in_place: unsafe fn(*mut ()),
    // methods shown in deterministic order
    foo_method: fn(*mut ()),
    bar_method: fn(*mut ()),
}

// fields contains Foo and Bar method addresses for u8 implementation
static VTABLE_FOO_FOR_U8: VTableFoo = VTableFoo { ... };

From the pseudo code, we can know

// let foo: &dyn Foo = &x;
let foo = TraitObjectFoo {&x, &VTABLE_FOO_FOR_U8};
// let bar: &dyn Bar = foo;
// C++ syntax for contructor
let bar = TraitObjectBar(TraitObjectFoo {&x, &VTABLE_FOO_FOR_U8});

The bar type is TraitObjectBar, which is not the type TraitObjectFoo. That is to say, you cannot assign a struct of one type to another different type (in rust, in C++ you can use reinterpret_cast).

What you can do it to have another level of indirection:

impl Bar for dyn Foo {
...
}

let bar: &dyn Bar = &foo;
// TraitObjectFoo {&foo, &VTABLE_FOO_FOR_DYN_FOO}

The same thing applies to Slice.

The workaround for casting different Unsized can be done by this trick:

// blanket impl for all sized types, this allows for a very large majority of use-cases
impl<T: Bar> AsBar for T {
    fn as_bar(&self) -> &dyn Bar { self }
}

// a helper-trait to do the conversion
trait AsBar {
    fn as_bar(&self) -> &dyn Bar;
}

// note that Bar requires `AsBar`, this is what allows you to call `as_bar`
// from a trait object of something that requires `Bar` as a super-trait
trait Bar: AsBar {
    fn bar_method(&self) {
        println!("this is bar");
    }
}

// no change here
trait Foo: Bar {
    fn foo_method(&self) {
        println!("this is foo");
    }
}

Answer 2

Not sure if that solves your concrete problem, but I did solve mine with the following trick:

I added the following method to FooTrait:

fn as_dyn(&self) -> &dyn FooTrait;

A default impl can not be provided (because it requires that Self be Sized, but constraining FooTrait to be Sized forbids creating trait objects for it...).

However, for all Sized implementations, it is trivially implemented as

fn as_dyn(&self) -> &dyn FooTrait { self }

So basically it constrains all implementations of FooTrait to be sized, except for dyn FooTrait.

Try it on the playground

Answer 3

This problem can be reduced to the following simple example (thanks to turbulencetoo):

trait Foo {}

fn make_dyn<T: Foo + ?Sized>(arg: &T) -> &dyn Foo {
    arg
}

At first glance, it really looks like this should compile, as you observed:

• If T is Sized, the compiler knows statically what vtable it should use to create the trait object;
• If T is dyn Foo, the vtable pointer is part of the reference and can just be copied to the output.

But there's a third possibility that throws a wrench in the works:

• If T is some unsized type that is not dyn Foo, even though the trait is object safe, there is no vtable for impl Foo for T.

The reason there is no vtable is because the vtable for a concrete type assumes that self pointers are thin pointers. When you call a method on a dyn Trait object, the vtable pointer is used to look up a function pointer, and only the data pointer is passed to the function.

However, suppose you implement a(n object-safe) trait for an unsized type:

trait Bar {}
trait Foo {
    fn foo(&self);
}

impl Foo for dyn Bar {
    fn foo(&self) {/* self is a fat pointer here */}
}

If there were a vtable for this impl, it would have to accept fat pointers, because the impl may use methods of Bar which are dynamically dispatched on self.

This causes two problems:

• There's nowhere to store the Bar vtable pointer in a &dyn Foo object, which is only two pointers in size (the data pointer and the Foo vtable pointer).
• Even if you had both pointers, you can't mix and match "fat pointer" vtables with "thin pointer" vtables, because they must be called in different ways.

Therefore, even though dyn Bar implements Foo, it is not possible to turn a &dyn Bar into a &dyn Foo.

Although slices (the other kind of unsized type) are not implemented using vtables, pointers to them are still fat, so the same limitation applies to impl Foo for [i32].

In some cases, you can use CoerceUnsized (only on nightly as of Rust 1.36) to express bounds like "must be coercible to &dyn FooTrait". Unfortunately, I don't see how to apply this in your case.

See also

• What is a "fat pointer" in Rust?
• Use trait object to pass str in rust has a concrete example of a reference to an unsized type (str) that cannot be coerced to a reference to a trait object.