Run script on iterator returned by `glob` with Rust's `run_cmd`

Question

Just a quick disclaimer to let you know that I'm a complete Rust illiterate. Taking that out of the way I can move to the question.

I'm trying to run a Python script on an iterator returned by glob with Rust's run_cmd. In other words, I'm trying to run a Python script which takes a file name and an integer as command line inputs and I wish to run this script only on files with a specific extension.

To grab the files with the extension I desire I'm using Rust's glob crate to return an iterator with the paths of these files. Having this iterator, I'm trying to run the following code to pass them to the Python script:

use cmd_lib::*;
use rayon::prelude::*;
use glob::glob;

fn main() {
    glob("../split_parsed/*.output")
        .into_par_iter()
        .for_each(|pattern|{
            run_cmd!(./src/aggregate.py ${pattern} 500).unwrap();
        });
}

Running this I get the following error:

error[E0599]: the method `to_string` exists for struct `Paths`, but its trait bounds were not satisfied
  --> src/main.rs:9:13
   |
9  |             run_cmd!(./src/aggregate.py ${pattern} 500).unwrap();
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ method cannot be called on `Paths` due to unsatisfied trait bounds
   | 
  ::: /home/ducbueno/.cargo/registry/src/github.com-1ecc6299db9ec823/glob-0.3.0/src/lib.rs:88:1
   |
88 | pub struct Paths {
   | ----------------
   | |
   | doesn't satisfy `Paths: ToString`
   | doesn't satisfy `Paths: std::fmt::Display`
   |
   = note: the following trait bounds were not satisfied:
           `Paths: std::fmt::Display`
           which is required by `Paths: ToString`
   = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

error: aborting due to previous error

For more information about this error, try `rustc --explain E0599`.
error: could not compile `aggregate`

Caused by:
  process didn't exit successfully: `rustc --crate-name aggregate --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=d0ac83c95bbb3012 -C extra-filename=-d0ac83c95bbb3012 --out-dir /hdd/mysrc/parse_gem5_debug/aggregate/target/debug/deps -C incremental=/hdd/mysrc/parse_gem5_debug/aggregate/target/debug/incremental -L dependency=/hdd/mysrc/parse_gem5_debug/aggregate/target/debug/deps --extern cmd_lib=/hdd/mysrc/parse_gem5_debug/aggregate/target/debug/deps/libcmd_lib-db4ce11b465defbd.rlib --extern glob=/hdd/mysrc/parse_gem5_debug/aggregate/target/debug/deps/libglob-99b91896bb92b644.rlib --extern rayon=/hdd/mysrc/parse_gem5_debug/aggregate/target/debug/deps/librayon-a4ecce8f6a92d238.rlib` (exit code: 1)

Does anyone know how I can fix this? Also any suggestions would be extremely welcome.

Thanks!

PS - Another quick disclaimer: I want to use Rust to execute this task due to the possibility of running the Python scripts in the files in parallel with Rayon. I had a great experience in the past using the parallel capabilities offered by Rayon in the past and would like to use it again.

Edit - full error message.

Answer 1

I recommend reading through the Error Handling chapter of the Rust Book, particularly the section that goes over Result.

The function call glob("../split_parsed/*.output") can fail (presumably due to I/O reasons) and therefore returns a Result. You're being fooled by .into_par_iter() here because Result is also an iterator (it yields a single element if successful, none otherwise). So what you get in your .for_each() is the full Paths object, which isn't what you want.

Here's a working sample:

fn main() {
    glob("../split_parsed/*.output")
        .expect("globbing failed")
        .map(|path| path.expect("globbing failed"))
        .par_bridge()
        .for_each(|path| {
            run_cmd!(./src/aggregate.py ${path} 500).unwrap();
        });
}

I use .expect() here to crash the program if the initial globbing or if the subsequent querying fails. Then use .par_bridge() instead of .into_par_iter() since Paths doesn't implement it. Hopefully this works for you.