return a closure that mutates its environment

Question

I'm writing a simple program that walks a directory, reading its entries and producing a JSON structure. I ran into trouble when I tried to return a closure that mutates a captured &mut Vec parameter:

use std::io;
use std::fs::{self, DirEntry,};
use std::path::Path;

extern crate rustc_serialize;
use rustc_serialize::json;


// json encoding: 
#[derive(Debug, RustcEncodable)]
struct Doc {
    path: String,
    filename: String,
}


fn main() {
    let target_path = Path::new("/Users/interaction/workspace/temp/testeddocs");
    let mut docs: Vec<Doc> = Vec::new();

    fn create_handler(docs: &mut Vec<Doc>) -> &FnMut(&DirEntry) {
        let handler = |entry: &DirEntry| -> () {
            let doc = Doc {
                path: entry.path().to_str().unwrap().to_string(),
                filename: entry.file_name().into_string().unwrap(),
            };
            docs.push(doc);
        };

        &handler
    }
    {
        let handler = create_handler(&mut docs);
        visit_dirs(&target_path, & |entry: &DirEntry|{
            handler(entry);
        });
    }
    println!("result json is: {}", json::encode(&docs).unwrap());
}

// one possible implementation of walking a directory only visiting files
fn visit_dirs(dir: &Path, cb: &Fn(&DirEntry)) -> io::Result<()> {
    if try!(fs::metadata(dir)).is_dir() {
        for entry in try!(fs::read_dir(dir)) {
            let entry = try!(entry);
            if try!(fs::metadata(entry.path())).is_dir() {
                try!(visit_dirs(&entry.path(), cb));
            } else {
                cb(&entry);
            }
        }
    }
    Ok(())
}

here's the compiler error it gives:

error: cannot borrow immutable borrowed content `***handler` as mutable
--> src/main.rs:36:13
   |
36 |             handler(entry);
   |             ^^^^^^^

error[E0373]: closure may outlive the current function, but it borrows `docs`, which is owned by the current function
--> src/main.rs:23:23
   |
23 |         let handler = |entry: &DirEntry| -> () {
   |                       ^^^^^^^^^^^^^^^^^^^^^^^^ may outlive borrowed value `docs`
...
28 |             docs.push(doc);
   |             ---- `docs` is borrowed here
   |
help: to force the closure to take ownership of `docs` (and any other referenced variables), use the `move` keyword, as shown:
|         let handler = move |entry: &DirEntry| -> () {

error: `handler` does not live long enough
--> src/main.rs:31:10
   |
31 |         &handler
   |          ^^^^^^^ does not live long enough
32 |     }
   |     - borrowed value only lives until here
   |
note: borrowed value must be valid for the anonymous lifetime #1 defined on the block at 22:64...
--> src/main.rs:22:65
   |
22 |     fn create_handler(docs: &mut Vec<Doc>) -> &FnMut(&DirEntry) {
   |

Answer 1

After days of fumbling, I think I have found a solution, by leveraging Box and move (move + Box don't create a deep clone).

But it is not good as I wanted, because I have to change visit_dirs signature (that piece of code is copied from rust doc, so I don't want change it). If anyone has a better suggestion, please let me know.

to @ker & @aochagavia, thanks for your help, it is truly appreciated.

use std::io;
use std::fs::{self, DirEntry,};
use std::path::Path;

extern crate rustc_serialize;
use rustc_serialize::json;


// json encoding: 
#[derive(Debug, RustcEncodable)]
struct Doc {
    path: String,
    filename: String,
}


fn main() {
    let target_path = Path::new("/Users/interaction/workspace/temp/testeddocs");
    let mut docs: Vec<Doc> = Vec::new();

    fn create_handler<'a>(docs: &'a mut Vec<Doc>) -> Box<FnMut(&DirEntry) + 'a> {
        let handler = move |entry: &DirEntry| -> () {
            let doc = Doc {
                path: entry.path().to_str().unwrap().to_string(),
                filename: entry.file_name().into_string().unwrap(),
            };
            docs.push(doc);
        };

        Box::new(handler)
    }

    {
        let mut handler = create_handler(&mut docs);
        visit_dirs(&target_path, &mut |entry: &DirEntry|{
            handler(entry)
        });
    }
    println!("result json is: {}", json::encode(&docs).unwrap());
}

// one possible implementation of walking a directory only visiting files
fn visit_dirs(dir: &Path, cb: &mut FnMut(&DirEntry)) -> io::Result<()> {
    if try!(fs::metadata(dir)).is_dir() {
        for entry in try!(fs::read_dir(dir)) {
            let entry = try!(entry);
            if try!(fs::metadata(entry.path())).is_dir() {
                try!(visit_dirs(&entry.path(), cb));
            } else {
                cb(&entry);
            }
        }
    }
    Ok(())
}

Answer 2

The problem

If you look carefully at create_handler, you will see that handler will be destroyed at the end of the function, since it is just a local variable. Therefore Rust forbids any references to handle that may be used from outside the function. Otherwise, the references would point to data that is no longer available (the classical dangling pointer bug).

Solution 1: boxing the closure

You can return the closure as a trait object by boxing it (allocating on the heap). This is the only way to do it in stable Rust (1.14):

fn create_handler(docs: &mut Vec<Doc>) -> Box<FnMut(&DirEntry)> {
    let handler = |entry: &DirEntry| -> () {
        let doc = Doc {
            path: entry.path().to_str().unwrap().to_string(),
            filename: entry.file_name().into_string().unwrap(),
        };
        docs.push(doc);
    };

    Box::new(handler)
}

Solution 2: returning the closure by value

While this does not work in stable Rust (1.14), you can use it on nightly. The benefit of this approach is that it avoids the heap allocation:

fn create_handler(docs: &mut Vec<Doc>) -> impl FnMut(&DirEntry) {
    let handler = |entry: &DirEntry| -> () {
        let doc = Doc {
            path: entry.path().to_str().unwrap().to_string(),
            filename: entry.file_name().into_string().unwrap(),
        };
        docs.push(doc);
    };

    handler
}