CODEWITHSUNDEEP
pythonmultithreadingasynchronousprawgrequests
ian-campbell
ian-campbell

Reputation: 1665

How can I execute batches of requests simultaneously in python?

I'm using the Reddit api to upvote some subreddits. I'd like to upvote multiple subreddits simultaneously, using the praw library to interact with the reddit api.

I want to upvote the 27 "hot" and "rising" posts of /r/bookclub:

subreddit = reddit.subreddit("bookclub")

for submission in subreddit.hot(limit=27):
    submission.upvote()

for submission in subreddit.rising(limit=27):
    submission.upvote()

I'd like to use async or multiprocessing to do this faster. I used the grequests library to do all the .upvote() at once, and it's working so far, but I'm not sure if it's really better:

subreddit = reddit.subreddit("bookclub")

hot_upvotes = (grequests.get(submission.upvote()) for submission in subreddit.hot(limit=27))
grequests.map(hot_upvotes)

rising_upvotes = (grequests.get(submission.upvote()) for submission in subreddit.rising(limit=27))
grequests.map(rising_upvotes)

Both versions upvote the subreddit. What I am wondering is if it is possible to run the hot_upvotes and rising_upvotes portions at the same time?

Upvotes: 1

Views: 723

Answers (1)

David Gomes
David Gomes

Reputation: 5825

from threading import Thread
import functools

subreddit = reddit.subreddit("bookclub")

def run_hot_upvotes(subreddit):
    hot_upvotes = (grequests.get(submission.upvote()) for submission in subreddit.hot(limit=27))
    grequests.map(hot_upvotes)

def run_rising_upvotes(subreddit):
    rising_upvotes = (grequests.get(submission.upvote()) for submission in subreddit.rising(limit=27))
    grequests.map(rising_upvotes)

Thread(target = functools.partial(run_rising_upvotes, subreddit)).start()
Thread(target = functools.partial(run_hot_upvotes, subreddit)).start()

Try the following to have both run at the same time, each in its own thread.

Upvotes: 1

Related Questions