CODEWITHSUNDEEP
wget
Piotr Kochański
Piotr Kochański

Reputation: 22672

How to skip selected url while mirroring site with wget

I have the following problem. I need to mirror password protected site. Sounds like simple task:

wget -m -k -K -E --cookies=on --keep-session-cookies --load-cookies=myCookies.txt http://mysite.com

in myCookies.txt I am keeping proper session cookie. This works until wget come accross logout page - then session is invalidated and, effectively, further mirroring is usless.

W tried to add --reject option, but it works only with file types - I can block only html file download or swf file download, I can't say

--reject http://mysite.com/*.php?type=Logout*

Any ideas how to skip certain URLs in wget? Maybe there is other tool that can do the job (must work on MS Windows).

Upvotes: 3

Views: 2687

Answers (2)

mythkiven
mythkiven

Reputation: 79

I have also encountered this problem and later solved it like this: "--reject-regex logout", more:wget-devTips

Upvotes: 3

Hans Lub
Hans Lub

Reputation: 5678

What if you first download (or even just touch) the logout page, and then

wget --no-clobber --your-original-arguments

This should skip the logout page, as it has already been downloaded

(Disclaimer: I didn't try this myself)

Upvotes: 4

Related Questions