Reputation: 912
I need to download many files from many website like e.g. 10 files which usually takes (8-9) seconds to download all the files.
What i want is to make the request simultaneously (parallel) so that the job finishes in (1-2) seconds.
I know use of curl in single requests and multiple requests. but the request of 10 files should start at the same time
e.g. 02:38:14 // all the request started at this time
I have a lot of bandwidth so parallel download will not be a problem.
is this possible in curl or by any other way ?
Upvotes: 0
Views: 747
Reputation: 614
If you want to do this in the shell, you can simply send a command to the background with &
. This should work:
for I in `cat urls.txt`; do curl $I & done
This loops through all urls in a file called urls.txt
and executes curl URL &
on them. The &
tells curl to run in the background.
Upvotes: 1