CODEWITHSUNDEEP
bashgnu-parallelgnome-terminal
PublisherName
PublisherName

Reputation: 131

How to use parallel with curl?

How do i use gnu parallel to make this process faster ?

#!/bin/bash
for (( c=1; c<=100; c++ ))
do  
    curl -sS 'https://example.com' \
        --data 'value='$c'' /dev/null
    echo $c
done

Upvotes: 1

Views: 2631

Answers (1)

dash-o
dash-o

Reputation: 14452

You can use parallel, or xargs

seq 100 | parallel curl -sS 'https://example.com' --data value='{}' /dev/null
seq 100 | xargs -I{} curl -sS 'https://example.com' --data value='{}' /dev/null

As the script stand, output will be sent to stdout. With xargs, this will result in output from different calls potentially mixed. Consider redirect output to files for additional processing, if needed.

You can add options for max parallel (-Pn, etc.) as needed

I'm not sure why '/dev/null' is needed. Consider reordering:

curl -sS --data value='{}' https://example.com'

Upvotes: 2

Related Questions