Type matching instead of `impl Trait` in trait definition

Question

These days, I am playing with Rust's async ecosystem, and Streams in particular, and I want to implement an equivalent of map() method (from StreamExt) with some specificities that are not interesting here. And because I am not allowed to write impl Stream as the return type of the extension trait I want to create, I feel quite limited in the choice of APIs I can define.

My code looks like that (I simplified the fn_to_future function):

use futures::{future::Ready, stream::Then, Stream, StreamExt};

pub fn fn_to_future<F, I, T>(f: F) -> impl Fn(I) -> Ready<T>
where
    F: Fn(I) -> T,
{
    // this is a reduced example
    move |input| {
        let out = f(input);
        futures::future::ready(out)
    }
}

// This is fine: this is a function, so we can use `impl`
// Also, it hides the actual type of the returned stream and
// allows for a change in the implementation without breaking the API.
pub fn my_map<S, F, T>(stream: S, f: F) -> impl Stream<Item = T>
where
    S: Stream,
    F: Fn(S::Item) -> T,
{
    stream.then(fn_to_future(f))
}

// This is the API I would like to offer to users, as it is
// easier to compose with existing functions
pub trait MyMapStreamExt: Stream + Sized {
    fn my_map_in_trait<F, T, G>(self, f: F) -> Then<Self, Ready<T>, G>
    // Cannot use 'impl Stream' here as we are in a trait
    // So, we try to be explicit about the return type 
    where
        F: Fn(Self::Item) -> T,
        G: Fn(Self::Item) -> Ready<T>,
    {
        self.then(fn_to_future(f)) // the compiler complains here 
        // as the type parameter 'G' cannot match the opaque type
        // returned by 'fn_to_future'
    }
}

I know that there are a few (possible) solutions to my particular problem :

1. Define a MyMap type that implements Stream, just as the Then type implements Stream.
2. Wrap the output of fn_to_future in a generic type parametrised by F and which implements Fn(Self::Item) -> Ready<T>.
3. Try using a Box as the output of the trait function (i.e. circumvent the impl Trait limitations).

Solution 1 will work for sure, but I feel like this is writing a lot of unnecessary code (cf. the amount of code used for the my_map function), and would mostly duplicate the Then implementation.

Solution 2 should work, but would require using unstable, which I would like to avoid, as there is no clear date of when the Fn traits will become stable (cf. the tracking issue).

I haven't really looked into Solution 3, but I find it very inelegant (and might cause many problems in my case).

So, my question is two-folds:

1. why does the type matching fail here (I don't quite understand if this is a weakness of the compiler or a soundness issue), and can we fix it?
2. is there any other elegant and efficient solution (as explained, I have either inelegant or inefficient solutions at the moment)?

Thanks!

Answer 1

The problem is by declaring the function like this:

fn my_map_in_trait<F, T, G>(self, f: F) -> Then<Self, Ready<T>, G>

Means that G is defined by the caller. This is definitely not the intention. Since you don't want to define a new Stream type, you need a named type instead of G. Here are some options for you:

1. Box the Fn

Implementation looks like this:

fn my_map_in_trait<'a, F, T>(self, f: F) -> Then<Self, Ready<T>, Box<dyn Fn(Self::Item) -> Ready<T> + 'a>>
where
    F: Fn(Self::Item) -> T,
    F: 'a,
    T: 'a,
    Self::Item: 'a,
{
    self.then(Box::new(fn_to_future(f)))
}

This should be better than Boxing the Stream implementation because less functions will need to be generated in the vtable.

2. existential types / TAIT (type alias impl trait)

An nightly only solution but doesn't have any overhead: playground

#![feature(type_alias_impl_trait)]
use futures::{future::Ready, stream::Then, Stream, StreamExt};

pub fn fn_to_future<S: Stream, F, T>(f: F) -> Fun<S, F, T>
where
    F: Fn(S::Item) -> T,
{
    move |input| {
        let out = f(input);
        futures::future::ready(out)
    }
}

type Fun<S: Stream, F, Output> = impl Fn(S::Item) -> Ready<Output>;

pub trait MyMapStreamExt: Stream + Sized {
    fn my_map_in_trait<'a, F, T>(self, f: F) -> Then<Self, Ready<T>, Fun<Self, F, T>>
    where
        F: Fn(Self::Item) -> T,
    {
        self.then(fn_to_future:<Self, _, _>(f))
    }
}

3. Unboxed closures

Another nightly only solution. This solution does not use impl Trait so no opaque types: playground

#![feature(fn_traits)]
#![feature(unboxed_closures)]

use futures::{future::Ready, stream::Then, Stream, StreamExt};

pub struct ToFuture<F>(F);

impl<F, I, O> FnOnce<(I,)> for ToFuture<F>
where
    F: FnOnce(I) -> O,
{
    type Output = Ready<O>;
    
    extern "rust-call" fn call_once(self, (i,): (I,)) -> Ready<O> {
        futures::future::ready((self.0)(i))
    }
}

impl<F, I, O> FnMut<(I,)> for ToFuture<F>
where
    F: FnMut(I) -> O,
{
    extern "rust-call" fn call_mut(&mut self, (i,): (I,)) -> Ready<O> {
        futures::future::ready((self.0)(i))
    }
}

pub fn fn_to_future<F>(f: F) -> ToFuture<F> {
    ToFuture(f)
}


pub trait MyMapStreamExt: Stream + Sized {
    fn my_map_in_trait<'a, F, T>(self, f: F) -> Then<Self, Ready<T>, ToFuture<F>>
    where
        F: Fn(Self::Item) -> T,
    {
        self.then(fn_to_future(f))
    }
}