CODEWITHSUNDEEP
bashshellwget
fragant
fragant

Reputation: 362

Download with wget and grep combined

I want to download a website with wget, then I want to extract a specific link from the website with the command grep and then I want to download this "grepped" link again with wget.

My attempt: 

wget -O website https://www.testwebsite.com/dir/site |
grep -E 'https://testwebsite.com/downloads/picture1' | wget -O myPicture

The problem is, that it is downloading the testwebsite.com/dir/site but doesn't extract the link and download it.

Upvotes: 2

Views: 3322

Answers (2)

u_Ltd.
u_Ltd.

Reputation: 544

wget has a recursion facility. You can invoke it with -r:

wget -r 'https://testwebsite.com'

(This works for links included in html)

The recursion depth can be set with -l, see man wget.

Upvotes: 1

dimm
dimm

Reputation: 1798

Try the -i - parametars on the last wget to read the links from stdin. Also the first wget should output to stdout with -O -

E.g.

wget -O - https://testwebsite.com | grep abcd | wget -i - -O outfile

Upvotes: 4

Related Questions