CODEWITHSUNDEEP
linuxbashshellwget
user3551932
user3551932

Reputation: 3

Shell script to download all webpages?

I am trying to download a range of pages from a site.

The URL would be in the format: http://example.com/x where x could be any number from 100 to 200.

Is there any possible script that I could use to download all of the pages, ranging from example.com/100 to example.com/200 ?

Upvotes: 0

Views: 117

Answers (3)

BeniBela
BeniBela

Reputation: 16907

Faster version that does not risk breaking more complex urls with incorrect quoting:

for x in {100..200}; do
    wget 'http://example.com/'"$x" &
done
wait

Upvotes: 0

gniourf_gniourf
gniourf_gniourf

Reputation: 46813

If you have curl installed, you might want to use the relatively efficient following way:

curl -O 'http://example.com/[100-200]'

(yes, I know, curl is really cool!).

Upvotes: 0

Barmar
Barmar

Reputation: 780688

for x in {100..200}; do
    wget "http://example.com/$x"
done

Upvotes: 3

Related Questions