CODEWITHSUNDEEP
bashcurl
Vlueboy
Vlueboy

Reputation: 172

curl to compile a list of redirected pages

Suppose I have a bash script that goes through a file that contains a list of old URLs that have all been redirected.

curl --location http://destination.com will process a page by following a redirect. However, I'm interested not in the content, but on where the redirect points so that I can update my records.

What is the command-line option for curl to output what that new location for the URL is?

Upvotes: 1

Views: 8390

Answers (3)

chown
chown

Reputation: 52778

You wou want to leave out the --location/-L flag, and use -w, checking the redirect_url variable. curl -w "%{redirect_url}" http://someurl.com should do it.

Used in a script:

REDIRECT=`curl -w "%{redirect_url}" http://someurl.com`
echo "http://someurl.com redirects to: ${REDIRECT}"

From the curl man page:

-w, --write-out <format>

Make curl display information on stdout after a completed transfer. The format is a string that may contain plain text mixed with any number of variables. The format can be specified as a literal "string", or you can have curl read the format from a file with "@filename" and to tell curl to read the format from stdin you write "@-".

The variables present in the output format will be substituted by the value or text that curl thinks fit, as described below. All variables are specified as %{variable_name} and to output a normal % you just write them as %%. You can output a newline by using \n, a carriage return with \r and a tab space with \t.

NOTE: The %-symbol is a special symbol in the win32-environment, where all occurrences of % must be doubled when using this option.

The variables available are:

...
redirect_url When an HTTP request was made without -L to follow redirects, this variable will show the actual URL a redirect would take you to. (Added in 7.18.2)
...

Upvotes: 8

ajreal
ajreal

Reputation: 47321

This might work (as a starting point)

curl -sI google.com | head -1 | grep 301 | wc -l

Upvotes: 2

Kent
Kent

Reputation: 195199

man curl

then

search redirect_url

redirect_url When a HTTP request was made without -L to follow redirects, this variable will show the actual URL a redirect would take you to. (Added in 7.18.2)

the variable above is for -w/--write-out <format>

Upvotes: 0

Related Questions