CODEWITHSUNDEEP
shellcurlwget
hendry
hendry

Reputation: 10813

wget syncing with changing remote HTTP files

I want to ensure an authorative remote file is in sync with a local file, without necessarily re-downloading the entire file.

I did mistakenly use wget -c http://example.com/filename

If "filename" was appended to remotely, that works fine. But if filename is prepended to, e.g. "bar" is prepended to a file just containing "foo", the end downloaded result filename contents in my test were wrongly "foo\nfoo", instead of "bar\nfoo".

Can anyone else suggest a different efficient http downloading tool? Something that looks at server caching headers or etags?

Upvotes: 2

Views: 4128

Answers (2)

thkala
thkala

Reputation: 86333

I believe that wget -N is what you are looking for. It turns on timestamping and allows wget to compare the local file timestamp with the remote timestamp. Keep in mind that you might still encounter corruption if the local file timestamp cannot be trusted e.g. if your local clock is drifting too much.

Upvotes: 4

mrbellek
mrbellek

Reputation: 2300

You could very well use curl: http://linux.about.com/od/commands/l/blcmdl1_curl.htm]1

Upvotes: 0

Related Questions