CODEWITHSUNDEEP
multithreadingrust
object Object
object Object

Reputation: 143

Unknown Size of HashMap | Passing Dynamic Functions into Multithreading

I've written a program to process my financial transactions but it's starting to run a little slower now that I'm adding more data to it. I decided to write it in Rust. I'm fluent in JS, TS, Python, bash/shell scripting. I need to feed the entire history into the application at this time. Currently my program is single-threaded. My thought is that if I use multi-threading strategically I should be able to cut the run-time down.

Here is how I attempted to implement multi-threading:

    for row in lines[1..].iter() {
        thread::spawn(|| {
            process_transaction(row, &rules) 
        });
    }

Rules is a hashmap that looks like this.

type CustomRule = Box<dyn Fn(&Transaction) -> &'static str>;
type RuleHashMap = HashMap<&'static str, CustomRule>;

Row is a Transaction structure w/ stuff and some functions implemented into it. (Regex, gt/lt matching) The key will be a regex string and the value will be that custom function. This custom function has to be put in a Box because it's size needs to go onto the heap? The idea is that I should be able to just quickly iterate over a set of regex patterns then run the corresponding additional logic needed to process that transaction. I am not mutating Transaction or Rules in any way and I'm only printing a result. Here's an example rule:

        rules.insert(r"(?i)7-ELEVEN|EXXONMOBIL|CIRCLE K|SUNOCO|SHEETZ|A-PLUS|RACEWAY|SHELLSERVICE|Shell SERVICE|QUICK NEASY|QUICK N EASY|FAS MART|BP|ROYAL MART|CITG|gas|wawa", Box::new(|t:&Transaction|{
            match t.less_than(15.0) {
                true => "expenses:convience",
                false => "expenses:business:gas"
            } 
        }));

The compiler suggested I update the type to implement Send

type CustomRule = dyn Fn(&Transaction) + Send + 'static;

I added that but now it says it does not know the size of the HashMap at compile time. I get this message from the compiler

= help: the trait `Sized` is not implemented for `(dyn for<'r> Fn(&'r Transaction) + Send + 'static)`
note: required by a bound in `HashMap`

What is this? I'm new to lower level programming like this and I want to understand what is actually happening versus just copying blindly. Especially when playing w/ threads. Was putting that custom rule function into a Box<> type the wrong move? Am I making this more complicated than needed.

Upvotes: 0

Views: 73

Answers (1)

PitaJ
PitaJ

Reputation: 15012

If all of your closures are "pure" functions that don't capture any state, then you can just store fn pointers in your map instead:

let mut rules: HashMap<&'static str, fn(&Transaction) -> &'static str> = HashMap::new();
rules.insert(r"some long regex", |t: &Transaction|{
    "expenses:convience"
});

You say

The idea is that I should be able to just quickly iterate over a set of regex patterns then run the corresponding additional logic needed to process that transaction.

If you're actually iterating, why even use a HashMap? Just use a Vec or array of tuples like so:

let mut rules: [(&'static str, fn(&Transaction) -> &'static str); 1] = [
    (r"some long regex", |t: &Transaction|{
        "expenses:convience"
    }),
];

You probably also should compile the Regexs only once, rather than compiling and running them every time. I recommend putting them in statics with once_cell::sync::Lazy.

Upvotes: 1

Related Questions