CODEWITHSUNDEEP
pythonbeautifulsoupwith-statement
Max Sebastian
Max Sebastian

Reputation: 15

Python Scraping Images With BeautifulSoup

I'm trying to scrape and download images from a website using BeautifulSoup. I have scraped a list of links stored in imgVal, the code can then create a new directory to store the images. The problem I have is the code will only download one image from the list of links. I would like to download them all. How can I accomplish this?

from bs4 import BeautifulSoup
from os.path import basename
import requests
import os

...

def writeImages():
    imgVal = getThumbnailLinks()
    imgBasename = '\n'.join(map(str, imgVal))

    pageNumber = '001'
    filename = pageNumber + '/'
    os.makedirs(os.path.dirname(filename), exist_ok=True)

    with open(filename + basename(imgBasename),"wb") as f:
        f.write(requests.get(imgBasename).content)

writeImages()

Upvotes: 1

Views: 1263

Answers (1)

ewwink
ewwink

Reputation: 19184

so you want to download multi line URL as input for requests? you can't, you have to do it one by one using loop.

def writeImages():
    pageNumber = '001'
    filename = pageNumber + '/'
    os.makedirs(os.path.dirname(filename), exist_ok=True)

    imgVal = getThumbnailLinks() # ['http://a.jpg', 'http://b.jpg']
    for imgBasename in imgVal:
        with open(filename + basename(imgBasename),"wb") as f:
            f.write(requests.get(imgBasename).content)

writeImages()

Upvotes: 1

Related Questions