How do I return an associated type from a higher-ranked trait bound trait?

Question

I have a trait that has a function for deserializing an associated type. However that associated type needs to have a lifetime that the caller decides, so I have a separate trait that I use a higher-ranked trait bound for, so that it can be deserialized for any lifetime.

I need to use a closure that returns this associated type.

I have the following code to do that:

#![allow(unreachable_code)]

use std::marker::PhantomData;

trait Endpoint: for<'a> EndpointBody<'a> {}
trait EndpointBody<'a> {
    type Out: 'a;
    fn serialize(body: &Self::Out) -> Vec<u8>;
    fn deserialize(raw_body: &'a [u8]) -> Self::Out;
}

// /////////////////////////////////////////////////////////

/// Trait object compatible handler
trait Handler {
    fn execute(&self, raw_body: &[u8]) -> Vec<u8>;
}

/// Wraps a function for an endpoint, convertint it to a Handler
struct FnHandler<EP, F>
where
    EP: Endpoint,
    F: 'static + for<'a> Fn(&'a [u8]) -> <EP as EndpointBody<'a>>::Out,
{
    func: F,
    _ph: PhantomData<EP>,
}
impl<EP, F> FnHandler<EP, F>
where
    EP: Endpoint,
    F: 'static + for<'a> Fn(&'a [u8]) -> <EP as EndpointBody<'a>>::Out,
{
    pub fn new(func: F) -> Self {
        Self {
            func,
            _ph: PhantomData,
        }
    }
}
impl<EP, F> Handler for FnHandler<EP, F>
where
    EP: Endpoint,
    F: 'static + for<'a> Fn(&'a [u8]) -> <EP as EndpointBody<'a>>::Out,
{
    fn execute(&self, in_raw_body: &[u8]) -> Vec<u8> {
        let body = (self.func)(in_raw_body);
        let serialized_body = unimplemented!();
        return serialized_body;
    }
}

// /////////////////////////////////////////////////////////

/// Collection of handlers
struct Handlers(Vec<Box<dyn Handler>>);
impl Handlers {
    pub fn new() -> Self {
        Self(vec![])
    }

    pub fn handle<EP: 'static, F>(&mut self, func: F)
    where
        EP: Endpoint,
        F: 'static + for<'a> Fn(&'a [u8]) -> <EP as EndpointBody<'a>>::Out,
    {
        self.0.push(Box::new(FnHandler::<EP, F>::new(func)));
    }
}

// /////////////////////////////////////////////////////////

struct MyEndpoint;
struct MyEndpointBody<'a> {
    pub string: &'a str,
}
impl Endpoint for MyEndpoint {}
impl<'a> EndpointBody<'a> for MyEndpoint {
    type Out = MyEndpointBody<'a>;

    fn serialize(body: &Self::Out) -> Vec<u8> {
        unimplemented!()
    }
    fn deserialize(raw_body: &'a [u8]) -> Self::Out {
        unimplemented!()
    }
}

// /////////////////////////////////////////////////////////

fn main() {
    let mut handlers = Handlers::new();
    handlers.handle::<MyEndpoint, _>(|_body| MyEndpointBody {
        string: "test string",
    });

    handlers.0[1].execute(&[]);
}

I think that should work, but when I check it I get a type error:

error[E0271]: type mismatch resolving `for<'a> <[closure@src/main.rs:92:38: 94:6] as std::ops::FnOnce<(&'a [u8],)>>::Output == <MyEndpoint as EndpointBody<'a>>::Out`
  --> src/main.rs:92:14
   |
92 |     handlers.handle::<MyEndpoint, _>(|_body| MyEndpointBody {
   |              ^^^^^^ expected struct `MyEndpointBody`, found associated type
   |
   = note:       expected struct `MyEndpointBody<'_>`
           found associated type `<MyEndpoint as EndpointBody<'_>>::Out`
   = note: consider constraining the associated type `<MyEndpoint as EndpointBody<'_>>::Out` to `MyEndpointBody<'_>`
   = note: for more information, visit https://doc.rust-lang.org/book/ch19-03-advanced-traits.html

It's confusing because MyEndpoint::Out is a MyEndpointBody, which I am returning from the closure, but Rust doesn't think they are the same type. I'm guessing it's because Rust picks incompatible anonymous lifetimes for the MyEndpointBody type, but I don't know how to fix that.

How can I get this code to work so that I can use a closure with a HRTB associated type?

Answer 1

EDIT: As of January 2024, the example in the question compiles successfully. It could also uses GATs to simplify the implementation.

Old answer:

---

Having the closure wrap the return type in a newtype fixes the issue:

#![allow(unreachable_code)]

use std::marker::PhantomData;

trait Endpoint: for<'a> EndpointBody<'a> {}
trait EndpointBody<'a> {
    type Out: 'a;
    fn serialize(body: &Self::Out) -> Vec<u8>;
    fn deserialize(raw_body: &'a [u8]) -> Self::Out;
}

struct EPOut<'a, EP: Endpoint>(<EP as EndpointBody<'a>>::Out);

// /////////////////////////////////////////////////////////

/// Trait object compatible handler
trait Handler {
    fn execute(&self, raw_body: &[u8]) -> Vec<u8>;
}

/// Wraps a function for an endpoint, convertint it to a Handler
struct FnHandler<EP, F>
where
    EP: Endpoint,
    F: 'static + for<'a> Fn(&'a [u8]) -> EPOut<'a, EP>,
{
    func: F,
    _ph: PhantomData<EP>,
}
impl<EP, F> FnHandler<EP, F>
where
    EP: Endpoint,
    F: 'static + for<'a> Fn(&'a [u8]) -> EPOut<'a, EP>,
{
    pub fn new(func: F) -> Self {
        Self {
            func,
            _ph: PhantomData,
        }
    }
}
impl<EP, F> Handler for FnHandler<EP, F>
where
    EP: Endpoint,
    F: 'static + for<'a> Fn(&'a [u8]) -> EPOut<'a, EP>,
{
    fn execute(&self, in_raw_body: &[u8]) -> Vec<u8> {
        let body = (self.func)(in_raw_body);
        let serialized_body = unimplemented!();
        return serialized_body;
    }
}

// /////////////////////////////////////////////////////////

/// Collection of handlers
struct Handlers(Vec<Box<dyn Handler>>);
impl Handlers {
    pub fn new() -> Self {
        Self(vec![])
    }

    pub fn handle<EP: 'static, F>(&mut self, func: F)
    where
        EP: Endpoint,
        F: 'static + for<'a> Fn(&'a [u8]) -> EPOut<'a, EP>,
    {
        self.0.push(Box::new(FnHandler::<EP, F>::new(func)));
    }
}

// /////////////////////////////////////////////////////////

struct MyEndpoint;
struct MyEndpointBody<'a> {
    pub string: &'a str,
}
impl Endpoint for MyEndpoint {}
impl<'a> EndpointBody<'a> for MyEndpoint {
    type Out = MyEndpointBody<'a>;

    fn serialize(body: &Self::Out) -> Vec<u8> {
        unimplemented!()
    }
    fn deserialize(raw_body: &'a [u8]) -> Self::Out {
        unimplemented!()
    }
}

// /////////////////////////////////////////////////////////

fn main() {
    let mut handlers = Handlers::new();
    handlers.handle::<MyEndpoint, _>(|_body| EPOut(MyEndpointBody {
        string: "test string",
    }));

    handlers.0[1].execute(&[]);
}

I'm tempted to say this is a Rust compiler bug, considering that the newtype should be just about the same as the associated type. There also seems to be some ICEs relating to using HRTB associated types: https://github.com/rust-lang/rust/issues/62529

Answer 2

I think the issue is that you request your handlers to be able to handle all possible lifetimes with that HK constraint - which the compiler cannot prove is verified, therefore not being able to make the equivalence MyEndpointBody <=> MyEndpoint::Out.

If instead, you parameterise your handlers to take a single lifetime, it appears to compile as required (playground link):

#![allow(unreachable_code)]

use std::marker::PhantomData;

trait Endpoint: for<'a> EndpointBody<'a> {}
trait EndpointBody<'a> {
    type Out: 'a;
    fn serialize(body: &Self::Out) -> Vec<u8>;
    fn deserialize(raw_body: &'a [u8]) -> Self::Out;
}
/// Trait object compatible handler
trait Handler<'a> {
    fn execute(&self, raw_body: &'a [u8]) -> Vec<u8>;
}

/// Wraps a function for an endpoint, convertint it to a Handler
struct FnHandler<EP, F>
where
    EP: Endpoint,
    F: 'static,
{
    func: F,
    _ph: PhantomData<EP>,
}
impl<EP, F> FnHandler<EP, F>
where
    EP: Endpoint,
    F: 'static,
{
    pub fn new(func: F) -> Self {
        Self {
            func,
            _ph: PhantomData,
        }
    }
}
impl<'a, EP, F> Handler<'a> for FnHandler<EP, F>
where
    EP: Endpoint,
    F: 'static + Fn(&'a [u8]) -> <EP as EndpointBody<'a>>::Out,
{
    fn execute(&self, in_raw_body: &'a [u8]) -> Vec<u8> {
        let body = (self.func)(in_raw_body);
        let serialized_body = unimplemented!();
        return serialized_body;
    }
}

// /////////////////////////////////////////////////////////

/// Collection of handlers
struct Handlers<'a>(Vec<Box<dyn Handler<'a>>>);
impl<'a> Handlers<'a> {
    pub fn new() -> Self {
        Self(vec![])
    }

    pub fn handle<EP: 'static, F>(&mut self, func: F)
    where
        EP: Endpoint,
        F: 'static + Fn(&'a [u8]) -> <EP as EndpointBody<'a>>::Out,
    {
        self.0.push(Box::new(FnHandler::<EP, F>::new(func)));
    }
}

// /////////////////////////////////////////////////////////

struct MyEndpoint;
struct MyEndpointBody<'a> {
    pub string: &'a str,
}
impl Endpoint for MyEndpoint {}
impl<'a> EndpointBody<'a> for MyEndpoint {
    type Out = MyEndpointBody<'a>;

    fn serialize(body: &Self::Out) -> Vec<u8> {
        unimplemented!()
    }
    fn deserialize(raw_body: &'a [u8]) -> Self::Out {
        unimplemented!()
    }
}

// /////////////////////////////////////////////////////////

fn main() {
    let mut handlers = Handlers::new();
    handlers.handle::<MyEndpoint, _>(|_body| MyEndpointBody {
        string: "test string",
    });

    handlers.0[1].execute(&[]);
}

Answer 3

Define DeserializeBody as:

trait DeserializeBody {
    type Out;
    fn deserialize(raw_body: &[u8]) -> Self::Out;
}

Out is a declaration of a generic type. Don't declare lifetime bound here, it will be explicit at the definition site.

At this point it is no more necessary the Higher-Rank Trait Bound for Endpoint:

trait Endpoint: DeserializeBody {}

trait DeserializeBody {
    type Out;
    fn deserialize(raw_body: &[u8]) -> Self::Out;
}

At the definition site a lifetime requirements must be expressed for the associated type Out. If DeserializeBody is not more a generic then MyEndpoint has to be:

impl<'a> DeserializeBody for MyEndpoint<'a> {
    type Out = MyEndpointBody<'a>;

    ...

And to implements such requirement lets resort to a phantom type that requires a lifetime 'a.

Putting all the pieces together:

use core::marker::PhantomData;

trait Endpoint: DeserializeBody {}

trait DeserializeBody {
    type Out;
    fn deserialize(raw_body: &[u8]) -> Self::Out;
}

fn store_ep<EP, F>(func: F)
where
    EP: Endpoint,
    F: 'static + for<'a> Fn(&'a [u8]) -> <EP as DeserializeBody>::Out,
{
    let _ = Box::new(func);
    unimplemented!();
}

struct MyEndpoint<'a> {
    phantom: PhantomData<&'a ()>
}

struct MyEndpointBody<'a> {
    pub string: &'a str,
}

impl<'a> Endpoint for MyEndpoint<'a> {}

impl<'a> DeserializeBody for MyEndpoint<'a> {
    type Out = MyEndpointBody<'a>;

    fn deserialize(raw_body: &[u8]) -> Self::Out {
        unimplemented!();
    }
}

fn main() {
    store_ep::<MyEndpoint, _>(|raw_body| MyEndpointBody { string: "test" });
}

Answer 4

Could you please check that one

trait Endpoint: for<'a> DeserializeBody<'a> {}
trait DeserializeBody<'a> {
    type Out: 'a;
    fn deserialize(raw_body: &'a [u8]) -> Self::Out;
}

fn store_ep<'a, EP, F>(func: F)
where
    EP: Endpoint,
    F: 'static + Fn(&'a [u8]) -> <EP as DeserializeBody<'a>>::Out,
{
    let _ = Box::new(func);
    unimplemented!();
}

// /////////////////////////////////////////////////////////

struct MyEndpoint;
struct MyEndpointBody<'a> {
    pub string: &'a str,
}
impl Endpoint for MyEndpoint {}
impl<'a> DeserializeBody<'a> for MyEndpoint {
    type Out = MyEndpointBody<'a>;
    fn deserialize(raw_body: &'a [u8]) -> Self::Out {
        unimplemented!();
    }
}

// /////////////////////////////////////////////////////////

fn main() {
    store_ep::<MyEndpoint, _>(|raw_body| MyEndpointBody { string: "test" });
}