CODEWITHSUNDEEP
pythonweb-scrapingreddit
Arturo
Arturo

Reputation: 3779

How to fetch more than 1000 images with the Reddit API?

I've written a Reddit image scraper and the Reddit API is only letting me go up to the latest 1000 images. Does anyone know if I can pay extra to get more data (to go into the hundred thousands), or a free workaround to this problem?

Upvotes: 1

Views: 1957

Answers (1)

bboe
bboe

Reputation: 4412

the reddit API is only letting me go up to the latest 1000 images

I assume you mean the listing only goes back 1000 submissions. That is a hard limit that applies to all reddit listings except for http://www.reddit.com/new (as far as I know). At the present time there is no option to pay for extra API access. However, assuming you obey reddit's API rules, you can use the http://www.reddit.com/new listing to go back as far as you want and simply filter out submissions to subreddits you don't care about.

Here's a simple PRAW example that counts the number of submissions made to a set of subreddits in the last 10000 submissions (it takes approximately 200 seconds to run).

import praw
r = praw.Reddit('stackoverflow question test')
subs = set(['aww', 'funny'])

count = 0
for submission in r.get_new(limit=10000):
    if submission.subreddit.display_name.lower() in subs:
        count += 1
print(count)

Upvotes: 4

Related Questions