CODEWITHSUNDEEP
bashxargs
Blaine
Blaine

Reputation: 2331

Copy line of text from input to output file while using xargs

I have a working shell script that reads a text file containing URLs, each on a separate line. URLs are read from the file in parallel and checked for their status code where the response is written to status-codes.csv.

How can I write the original URL referenced from url-list.txt to the first column of output in status-codes.csv?

status-codes.sh

#!/bin/bash
xargs -n1 -P 10 curl -u user:pass -L -o /dev/null --silent --head --write-out '%{url_effective},%{http_code},%{num_redirects}\n' < url-list.txt | tee status-codes.csv

url-list.txt

http://website-url.com/path-to-page-1
http://website-url.com/path-to-page-2
http://website-url.com/path-to-page-3

status-codes.csv (Current Output)

http://website-url.com/path-to-page-2,200,1
http://website-url.com/path-to-page-after-any-redirects,200,2
http://website-url.com/404,404,2

status-codes.csv (Desired Output)

http://website-url.com/path-to-page-2,http://website-url.com/path-to-page-2,200,1
http://website-url.com/path-to-page-1,http://website-url.com/path-to-page-after-any-redirects,200,2
http://website-url.com/path-to-page-3,http://website-url.com/404,404,2

Upvotes: 3

Views: 216

Answers (1)

oguz ismail
oguz ismail

Reputation: 50785

Use -I option. E.g:

xargs -n1 -P 10 -I '{}' curl -u user:pass -L -o /dev/null --silent --head --write-out '{},%{url_effective},%{http_code},%{num_redirects}\n' '{}' < url-list.txt | tee status-codes.csv

man xargs:

-I replace-str Replace occurrences of replace-str in the initial-arguments with names read from standard input.

Upvotes: 2

Related Questions