CODEWITHSUNDEEP
python-3.xprawreddit
jarcobi889
jarcobi889

Reputation: 845

PRAW Requesting All Subreddit Posts: Receive 401 Error

I'm trying to grab every single post from a subreddit all the way back to it's beginning in 2010, specifically /r/nosleep.

My code for grabbing those posts is the usual: 

for submission in nosleep.submissions(end=int(time.time()):

It works perfectly, I've checked my credentials they all work, and it will easily grab two years worth of posts without any issue. What happens is I run the above for loop, and at some point around the end, it returns a 401 and crashes the entire program.

I've checked and confirmed the following scenarios:

The only "hack" around this is to split up the work into two for loops, splitting int(time.time()) into two (or more) pieces and iterating over each like this: 

for submission in nosleep.submissions(start=middle, end=int(time.time())):
for submission in nosleep.submissions(end=middle):

Even then, it sometimes returns a 401. I suspect it's because of the length of time this loop is running, but I don't know. Does anyone have any suggestions for a new method, or editing the PRAW source to accommodate?

Upvotes: 2

Views: 1292

Answers (2)

Dr-Bracket
Dr-Bracket

Reputation: 5594

For future Googlers, this issue can also be caused after getting the password wrong multiple times. It takes about 20 minutes to reset, then it will work again.

Upvotes: 1

bboe
bboe

Reputation: 4422

Try the latest development version of PRAW (pip install --upgrade https://github.com/praw-dev/praw/archive/master.zip) as this issue should be resolved.

Upvotes: 1

Related Questions