CODEWITHSUNDEEP
pythonweb-scrapingpython-requests
user8018985
user8018985

Reputation:

Python requests giving errror: IndexError: list index out of range

Python requests giving error: IndexError: list index out of range :

import os
import csv
import requests

write_path = '/Users/specter/Desktop/pdfs/u'  # ASSUMING THAT FOLDER EXISTS!

with open('final.csv', 'r') as csvfile:
    spamreader = csv.reader(csvfile)
    for link in spamreader:
        print('-'*72)
        pdf_file = link[0].split('/')[-1]
    with open(os.path.join(write_path, pdf_file), 'wb') as pdf:
        try:
            # Try to request PDF from URL
            print('TRYING {}...'.format(link[0]))
            a = requests.get(link[0], stream=True)
            for block in a.iter_content(512):
                if not block:
                    break

                pdf.write(block)
            print('OK.')
        except requests.exceptions.RequestException as e:  # This will catch ONLY Requests exceptions
            print('REQUESTS ERROR:')
            print(e)  # This should tell you more details about the error

While trying to download 1000+ pdf's files using request package in python.

Traceback (most recent call last):
  File "update.py", line 11, in <module>
    pdf_file = link[0].split('/')[-1] 
IndexError: list index out of range

Error

Upvotes: 0

Views: 1274

Answers (1)

JohanL
JohanL

Reputation: 6891

There are probably some empty lines in your csv file. In that case link will be the empty string '' and you will get an index error. Change the code to:

.
.
.
with open('final.csv', 'r') as csvfile:
    spamreader = csv.reader(csvfile)
    for link in spamreader:
        if not link:
            continue
        print('-'*72)
        pdf_file = link[0].split('/')[-1]
.
.
.

On a further note; your code seems to be strangely indented. As it stands, it will only open the last pdf in final.csv. Are you sure you do not want to indent your second with statement, together with the rest of the code, one more level, to be executed within the for loop?

Upvotes: 1

Related Questions