Can I make this struct more generic?

Question

Playground

Context

I want to pre-process the lines of a text file before passing them to a function f. I can do it like this:

pub fn example0a<B: BufRead, F: Fn(&str)>(bufread: B, f: F) {
    let name = Regex::new("John Doe").unwrap();

    for line in bufread.lines() {
        let line = line.unwrap();
        let pre_processed_line = name.replace_all(&line, "XXX");
        f(&pre_processed_line);
    }
}

but I need to produce an object, with a for_each method, to which I could pass f directly. My first idea was to produce an iterator using the map method:

// does not compile
pub fn example0b<B: BufRead>(bufread: B) -> impl Iterator {
    let name = Regex::new("John Doe").unwrap();

    bufread.lines().map(move |line| {
        let line = line.unwrap();
        let pre_processed_line = name.replace_all(&line, "XXX");
        &pre_processed_line as &str;
    })
}

This does not compile because line and hence pre_processed_line do not live long enough to be returned out of the iterator's next method. One option is to return pre_processed_line.to_string(), but that is not good, because it clones all the lines that have not been modified by replace_all, which I want to avoid.

My first struct

I decided to implement a struct containing the BufRead and the pre-processing function, and providing a for_each method. I aimed to make it as generic as possible, so in fact it accepts any iterator of any type, as long as that the pre-processing function can transform it to a &str.

pub struct TransformedStrStream<S, FT>
where
    S: Iterator,
    FT: FnMut(S::Item, &mut FnMut(&str)),
{
    source: S,
    transf: FT,
}

impl<S, FT> TransformedStrStream<S, FT>
where
    S: Iterator,
    FT: FnMut(S::Item, &mut FnMut(&str)),
{
    pub fn for_each<F>(self, mut f: F)
    where
        F: FnMut(&str),
    {
        let source = self.source;
        let mut transf = self.transf;
        source.for_each(move |line| transf(line, &mut f));
    }
}

I can instantiate that struct in a way similar to the examples above:

pub fn example1<B: BufRead>(bufread: B, name: Regex) {
    let _ = TransformedStrStream {
        source: bufread.lines(),
        transf: move |line, f| {
            let line = line.unwrap();
            let repl = name.replace_all(&line, "XXX");
            f(&repl as &str)
        },
    };
}

My problem

The struct above is, I think, a nice abstraction, and could be abstracted even further, to generate any type of value (rather than &str).

I tried to replace &str by a type parameter T:

pub struct TransformedStream<S, FT, T>
where
    S: Iterator,
    FT: FnMut(S::Item, &mut FnMut(T)),
{
    source: S,
    transf: FT,
    phantom: PhantomData<T>,
}

impl<S, FT, T> TransformedStream<S, FT, T>
where
    S: Iterator,
    FT: FnMut(S::Item, &mut FnMut(T)),
{
    pub fn for_each<F>(self, mut f: F)
    where
        F: FnMut(T),
    {
        let source = self.source;
        let mut transf = self.transf;
        source.for_each(move |line| transf(line, &mut f));
    }
}

Unfortunately, my example above does not compile anymore:

pub fn example2<B: BufRead>(bufread: B, name: Regex) {
    let _ = TransformedStream {
        source: bufread.lines(),
        transf: move |line, f| {
            let line = line.unwrap();
            let repl = name.replace_all(&line, "XXX");
            f(&repl as &str)
        },
        phantom: PhantomData,
    };
}

error[E0597]: `line` does not live long enough
  --> src/lib.rs:37:42
   |
37 |             let repl = name.replace_all(&line, "XXX");
   |                                          ^^^^ borrowed value does not live long enough
38 |             f(&repl as &str)
39 |         },
   |         - `line` dropped here while still borrowed
40 |         phantom: PhantomData,
41 |     };
   |     - borrowed value needs to live until here

error[E0597]: `repl` does not live long enough
  --> src/lib.rs:38:16
   |
38 |             f(&repl as &str)
   |                ^^^^ borrowed value does not live long enough
39 |         },
   |         - `repl` dropped here while still borrowed
40 |         phantom: PhantomData,
41 |     };
   |     - borrowed value needs to live until here

In my opinion, line and repl live long enough to be processed by f, just as in the &str version. As for the iterator example above, using repl.to_string() satisfies the compiler, but I don't want to clone each and every line.

My intuition is that the problem comes from the PhantomData<T> that I had to add to my struct to satisfy the compiler. Does it constrain the lifetime of T (to live as long as the containing struct), just as if I had a field with type T? I tried to replace it with PhantomData<*const T>, which I thought might not constrain the lifetime, but it does not solve my problem...

Why does the second version not compile? How I could make it work?

Answer 1

Instead of T, use &T. It's a more direct translation of the original &str version, so you can be more sure it will work after making the change. And it does:

pub struct TransformedStream<S, FT, T>
where
    S: Iterator,
    T: ?Sized,
    FT: FnMut(S::Item, &mut FnMut(&T)),
{
    source: S,
    transf: FT,
    phantom: PhantomData<*const T>,
}

impl<S, FT, T> TransformedStream<S, FT, T>
where
    S: Iterator,
    T: ?Sized,
    FT: FnMut(S::Item, &mut FnMut(&T)),
{
    pub fn for_each<F> (self, mut f: F) where F: FnMut(&T) {
        let source = self.source;
        let mut transf = self.transf;
        source.for_each(move |line| transf(line, &mut f));
    }
}