CODEWITHSUNDEEP
downloadwget
Paul S.
Paul S.

Reputation: 4492

recursive wget control download

The recursive option of wget (-r option) occasionally yields many files. For example,wget -r www.cnn.com gives the following:

--2013-05-05 10:35:54--  http://www.cnn.com/about/
Reusing existing connection to www.cnn.com:80.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘www.cnn.com/about/index.html’

--2013-05-05 10:35:54--  http://www.cnn.com/help/
Reusing existing connection to www.cnn.com:80.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to: ‘www.cnn.com/help/index.html’

......

How can this be controlled - can we to specify the maximum total size of download, the maximum time of download, etc.?

Upvotes: 1

Views: 268

Answers (1)

vkoe
vkoe

Reputation: 381

To stop a command after a certain time you can use the command timeout.

timeout 60 wget ...

will stop the download after 60 seconds.

You can add an optional unit after the duration:

  • 's' for seconds (default)
  • 'm' for minutes
  • 'h' for hours
  • 'd' for days

Upvotes: 1

Related Questions