CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Sean Kelly
Sean Kelly

Reputation: 183

(Short) - POST data should be bytes or an iterable of bytes. It cannot be of type str

Just trying to feed-in links from a .csv file, then scrape info from each link, then write it to other columns in the .csv. I've been scratching my head for days. Can anyone else see what's wrong here? The error happens at soup

def scrape_data(csv_file):
writer = csv.writer(csv_file)
reader = csv.reader(csv_file)  

for row in reader:
    if row:

        # THE ERROR HAPPENS AT THE SOUP OBJECT BELOW            

        soup = BeautifulSoup(urllib.request.urlopen(row[0], 'lxml'))
        post_time = soup.find('time', {'class' : 'date timeago'})            
        sqfeet = (sqft.text for sqft in soup.find('span', {'class' : 'shared-line-bubble'})) 
        availability = (soup.find('span', {'class' : 'data-date'}))            
        attribute_group = (ag.text for ag in soup.find('p', {'class' : 'attrgroup'}))
        address = (add.text for add in soup.find('div', {'class' : 'mapaddress'}))            

        for data in zip(post_time, sqfeet, availability, attribute_group, address):
            writer.writerow(row[3])

Upvotes: 0

Views: 79

Answers (1)

Michael Butscher
Michael Butscher

Reputation: 10959

The 'lxml' part must be a parameter of BeautifulSoup() but is parameter of urllib.request.urlopen()

Upvotes: 3

Related Questions