CODEWITHSUNDEEP
pythonwget
ctdeveloper
ctdeveloper

Reputation: 290

Python - wget check when process has completed

I have a function that is calling wget via subprocess.Popen. The purpose of this function is to spawn wget and spider a website for a list of links.

Is it possible to tell when the wget process has completed and then continue executing the rest of the python function e.g.

def get_urls(url, uname, pword, output):
    subprocess.Popen (['wget', '-nd', '-r', '--user=', uname, '--password=', pword,  
    '--no-parent','--spider',url, '--output-file= ',output], stdout=subprocess.PIPE)

    #some method telling wget has finished writing to the output file, so continue

    foo = bar() #rest of function etc.

Also is there a better method of spidering a site (and passing in login credentials) via python rather than making system calls?

Thanks

Upvotes: 0

Views: 1508

Answers (3)

Flávio Fonseca
Flávio Fonseca

Reputation: 338

When you call subprocess.Popen it creates a new process that is run in background and code code keeps running.

If you want to wait for the created process to finish just use Popen.wait method.

new_process = subprocess.Popen(...)
new_process.wait() # waits for the process to finish. you can also pass a timeout parameter
foo_bar() # do whatever after download is finished

Upvotes: 0

pod2metra
pod2metra

Reputation: 266

Why you want use subprocess, may be better use urllib

import urllib

url = 'http:......'
filename = 'your_filename'
urllib.urlretrieve(url, filename)

Upvotes: 0

alan
alan

Reputation: 4842

Perhaps you could use subprocess.call or subprocess.check_call instead? They will both wait for a command to complete, then give you the return code.

See doc here

Upvotes: 1

Related Questions