CODEWITHSUNDEEP
bashcurl
Antony
Antony

Reputation: 4310

Loop through CURL results

I have the following script:

#!/bin/bash
for line in $('curl -s https://scans.io/data/rapid7/sonar.http/20141209-http.gz | zcat | head -n 1000 | dap json + select vhost + lines');
do
    echo "$line\n"
done

For which I am trying to achieve the following:

  1. Loop through a dynamically growing list of results from the scans.io curl
  2. Output the line which I then propose to pass to PHP to store and process

However I'm getting syntax error near unexpected token$'\r''` but am not a BASH expert to know what I need to tweak.

Upvotes: 3

Views: 6586

Answers (1)

Inian
Inian

Reputation: 85530

Use Process-Substitution, with a while-loop, see why using for-loop for command output parsing is bad. The unsetting of IFS and read with the -r flag does not allow backslashes to escape any characters and treat strings "as-is".

#!/bin/bash

while IFS= read -r line
do
    printf '%s\n' "$line"
done < <(curl -s https://scans.io/data/rapid7/sonar.http/20141209-http.gz | zcat | head -n 1000 | dap json + select vhost + lines)

Upvotes: 5

Related Questions