CODEWITHSUNDEEP
curl
KolonUK
KolonUK

Reputation: 503

Show the final redirect headers using curl

The curl command:

curl -IL "http://www.bbc.co.uk"

returns:

HTTP/1.1 301 Moved Permanently
Server: nginx
X-BBC-No-Scheme-Rewrite: 1
X-Cache-Action: HIT
X-Cache-Hits: 18686
Vary: X-BBC-Edge-Scheme
Cache-Control: public, max-age=3600
X-Cache-Age: 2624
Content-Type: text/html
Date: Thu, 02 Nov 2017 16:55:53 GMT
Location: https://www.bbc.co.uk/
Content-Length: 178
Connection: Keep-Alive

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
ETag: W/"3fdf4-XNfGeb/Aqcmv8OYZKVQdoUFYC34"
X-Frame-Options: SAMEORIGIN
Content-Length: 261620
Date: Thu, 02 Nov 2017 16:55:53 GMT
Connection: keep-alive
Set-Cookie: BBC-UID=65699f7bb41e91e9d7e103d8618f7780367bf9e637843476ca70678244f407000curl/7.19.7%20(x86_64-redhat-linux-gnu)%20libcurl/7.19.7%20NSS/3.19.1%20Basic%20ECC%20zlib/1.2.3%20libidn/1.18%20libssh2/1.4.2; expires=Mon, 01-Nov-21 16:55:53 GMT; path=/; domain=.bbc.co.uk
X-Cache-Action: HIT
X-Cache-Hits: 5
X-Cache-Age: 0
Cache-Control: private, max-age=0, must-revalidate
Vary: Accept-Encoding, X-CDN, X-BBC-Edge-Scheme

(In production, I use -i and not -I to get the whole payload)

How do I get curl to hide the preceding header(s) that issue the redirect, and just return the header of the final target resource?

Upvotes: 3

Views: 7894

Answers (1)

randomir
randomir

Reputation: 18697

There's no such an option in curl, but this simple sed program is all you need:

sed '/^HTTP\/1.1 3[0-9][0-9]/,/^\r$/d' file

If you pipe your curl output through it, it will delete (notice the d action at the end) all header blocks that start with any 3xx redirection code (HTTP/1.1 3[0-9][0-9]) and end with an empty line (an empty line in Unix is actually \n only, recognized in sed with ^$ pattern, but HTTP headers use DOS newline sequence \r\n, which after stripping by sed becomes \r; so we have to match ^\r$). To select a range of lines, sed range addressing is used with regexp addresses.

For your example:

$ curl -sIL "http://www.bbc.co.uk" | sed '/^HTTP\/1.1 3[0-9][0-9]/,/^\r$/d'
HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
ETag: W/"3fbed-SZN5fj1ugmv3uhAJWfQBrAvzK1M"
X-Frame-Options: SAMEORIGIN
Content-Length: 261101
Date: Thu, 02 Nov 2017 22:53:19 GMT
Connection: keep-alive
Set-Cookie: BBC-UID=95994f2b2a4bf...57f26d5442240curl/7.50.1; expires=Mon, 01-Nov-21 22:53:19 GMT; path=/; domain=.bbc.co.uk
X-Cache-Action: HIT
X-Cache-Hits: 649
X-Cache-Age: 119
Cache-Control: private, max-age=0, must-revalidate
Vary: Accept-Encoding, X-CDN, X-BBC-Edge-Scheme

Similarly for the complete response (with body):

$ curl -siL "http://www.bbc.co.uk" | sed '/^HTTP\/1.1 3[0-9][0-9]/,/^\r$/d'

Notice the -s flag we used above to suppress curl's progress output.

Upvotes: 3

Related Questions