CODEWITHSUNDEEP
pythonterminalwget
Andy
Andy

Reputation: 263

Executing wget from python

Heyo, trynna download images from a site. I've setup a basic filter which works fine but my aim is to automate this and one of the steps to doing that is constantly re-downloading the site. I'm using wget to do this which works fine from terminal but it seems os.system() in python creates it's own (can't think of the name atm) 'terminal' which means I can't use things that I've installed, such as wget. I've tried gnome-terminal but I might be doing something wrong :/ Any other solutions would be greatly appreciated, thanks!

Upvotes: 0

Views: 414

Answers (1)

George TG
George TG

Reputation: 178

Why are you trying to download the site by calling wget from the terminal ? I think a better idea is to download a site the python way:

import sys
import os
import urllib.error
import urllib.request

def get_raw_webpage(url):
    """
        Download a web url as raw bytes
    """
    try:
        req = urllib.request.Request(url)
        response = urllib.request.urlopen(req)
        data = response.read()
        return data

    except urllib.error.HTTPError as e:
        print('HTTPError: ', e.code , file = sys.stderr)
        return None

    except urllib.error.URLError as e:
        print('URLError: ', e.args, file = sys.stderr)
        return None

    except ValueError as e:
        print('Invalid url.', e.args, file = sys.stderr)

    return None


def get_webpage(url):
    """
    Get webpage as raw bytes and then
    convert to readable form
    """
    data = get_raw_webpage(url)
    if data == None:
        return None

    return data.decode('utf-8')

You can also use get_raw_webpage function with a link to an image to download it!

Upvotes: 1

Related Questions