CODEWITHSUNDEEP
pythonhtmlpython-3.ximage
test123
test123

Reputation: 127

Finding and downloading images from websites using urllib2

I'm writing a small program that given an http address, it will find and download the images within. My current situation is this:

import urllib2, html

class HTMLNode(object):
    def __init__(self,tag,attr,content,closed=True):
        self.tag = tag
        self.attr = attr
        self.content = content
        self.closed = closed

    def istext(self):
        return self.tag == '_text_'

    def tostring(self):
        if self.istext(): 
            return self.content
        ret = '<'+self.tag
        for k, v in self.attr.items():
            ret += ' '+k+'="'+v+'"'
        ret += '>'
        if self.closed:
            for c in self.content:
                ret += c.tostring()
            ret += '</'+self.tag+'>'
        return ret

    def find_by_tag(self,tag):
            ret = []
            if self.tag == tag: ret += [self]
            if not self.istext():
                for c in self.content:
                    ret += c.find_by_tag(tag)
            return ret

def imagegrab(url):
    req = urllib2.Request(url)
    response = urllib2.urlopen(req)
    output = open(url, 'wb')
    output.write(response.read())

The missing link is finding a way to use the "find_by_tag" function of the HTMLNode class so that the program will scan for the "" tag and download the pics from the website. Can anybody help me with that?

Upvotes: 0

Views: 139

Answers (1)

Katriel
Katriel

Reputation: 123782

Other people have already done this work for you. Specifically, take a look at BeautifulSoup, which is an HTML parsing library for Python. You would do

soup = bs4.BeautifulSoup(...)
for img in soup("img"):
    print img.src

or similar; obviously the library is much more powerful than that.

You could also consider using requests, which is a wrapper for the urllib family of libraries with a beautifully simple API. For that you would do

soup = bs4.BeautifulSoup(requests.get(url).text)

Upvotes: 2

Related Questions