CODEWITHSUNDEEP
rustureq
ditoslav
ditoslav

Reputation: 4872

How to execute a map of blocking http requests in parallel?

I have a lot of code using ureq for http requests and I'm wondering if I can avoid using another http library.

I have a list of urls and I'm invoking ureq::get on them. I'm wondering if I can somehow make these calls in parallel. How would I create separate threads and execute these in parallel?

let v = vec!["www.qwe.com"; 3];
let responses = v.into_iter().map(|s| make_get_request(s)).collect::<Vec<_>>();

Upvotes: 0

Views: 538

Answers (2)

Jmb
Jmb

Reputation: 23274

If you want as many threads as requests, you can just spawn them:

let v = vec!["www.qwe.com"; 3];
let handles = v
    .into_iter()
    .map(|s| thread::spawn(move || make_get_request(s)))
    .collect::<Vec<_>>();
let responses = handles.into_iter().map(|h| h.join()).collect::<Vec<_>>();

Playground

Upvotes: 0

Masklinn
Masklinn

Reputation: 42272

You can just use rayon. It's not ideal because it assumes CPU-bound work and will therefore spawn one thread per (logical) core by default, which may be less than you'd want for HTTP requests, but you could always customise the global threadpool (or run your work inside the scope of a local threadpool with a higher thread count).

Upvotes: 2

Related Questions