Skip download if files already exist in wget?

Question

Here is a simple wget command:

wget http://www.example.com/images/misc/pic.png

How to make wget skip download if pic.png is already available ?

Answer 1

I had issues with -N as I wanted to save output to a different file name.

Timestamping, wget docs:

A file is considered new if one of these two conditions are met:

1. A file of that name does not already exist locally.
2. A file of that name does exist, but the remote file was modified more recently than the local file.

Using test:

test -f stackoverflow.html || wget -O stackoverflow.html https://stackoverflow.com/

If the file exists does not exist test will evaluate to FALSE so wget will be executed.

Answer 2

The -nc, --no-clobber option isn't the best solution as newer files will not be downloaded. One should use -N instead which will download and overwrite the file only if the server has a newer version, so the correct answer is:

wget -N http://www.example.com/images/misc/pic.png

Then running Wget with -N, with or without -r or -p, the decision as to whether or not to download a newer copy of a file depends on the local and remote timestamp and size of the file. -nc may not be specified at the same time as -N.

-N, --timestamping: Turn on time-stamping.

Answer 3

The answer I was looking for is at https://unix.stackexchange.com/a/9557/114862.

Using the -c flag when the local file is of greater or equal size to the server version will avoid re-downloading.

Answer 4

When running Wget with -r or -p, but without -N, -nd, or -nc, re-downloading a file will result in the new copy simply overwriting the old.

So adding -nc will prevent this behavior, instead causing the original version to be preserved and any newer copies on the server to be ignored.

See more info at GNU.

Answer 5

Try the following parameter:

-nc, --no-clobber: skip downloads that would download to existing files.

Sample usage:

wget -nc http://example.com/pic.png