CODEWITHSUNDEEP
phpasynchronousguzzlehttp-proxy
Fuad
Fuad

Reputation: 1102

How to send asynchronous requests by guzzle without using proxy

I am trying to send 100 requests to instagram website asynchronously by php guzzle like this:

$client = new Client();
$promises = [];
for($i = 1; $i <= 100; $i++)
{
    $options = ['timeout' => 60];

    $promise = $client->getAsync("https://www.instagram.com/p/B7IoQYhACcF/", $options)
            ->then(function($response){
                return $response;
            }, function($exception){
                return [
                    'msg' => $exception->getMessage(),
                ];
            });   
    $promises[$i] = $promise;
}

after that I try to print requests status:

foreach ($promises as $key => $promise)
{
    $response = $promise->wait();

    if(is_array($response))
    {
        echo "{$key}) error: {$response['msg']} <br/>";
        continue;
    }

    echo "{$key}) {$response->getStatusCode()} {$response->getReasonPhrase()} <br/>";
}

ok! but what is the problem here?

The problem is that only the first 20 (or some more) requests are ok and other requests not resolved.

the result of this code is here:

1) 200 OK
2) 200 OK
3) 200 OK
4) 200 OK
5) 200 OK
6) 200 OK
7) 200 OK
8) 200 OK
9) 200 OK
10) 200 OK
11) 200 OK
12) 200 OK
13) 200 OK
14) 200 OK
15) 200 OK
16) 200 OK
17) 200 OK
18) 200 OK
19) 200 OK
20) 200 OK
21) 200 OK
22) 200 OK
23) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
24) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
25) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
26) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
27) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
28) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
29) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
30) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
31) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
32) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
33) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
34) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
35) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
36) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
37) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
38) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
39) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
40) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
41) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
42) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
43) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
44) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
45) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
46) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
47) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
48) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
49) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
50) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
51) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
52) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
53) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
54) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
55) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
56) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
57) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
58) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
59) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
60) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
61) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
62) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
63) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
64) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
65) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
66) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
67) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
68) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
69) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
70) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
71) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
72) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
73) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
74) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
75) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
76) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
77) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
78) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
79) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
80) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
81) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
82) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
83) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
84) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
85) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
86) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
87) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
88) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
89) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
90) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
91) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
92) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
93) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
94) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
95) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
96) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
97) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
98) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
99) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)
100) error: cURL error 6: (see https://curl.haxx.se/libcurl/c/libcurl-errors.html)

note that I resolved this problem by using an http proxy but i cant use any proxies for doing that.

$options = ['timeout' => 60, 'proxy' => '151.253.165.70:8080'];

I think when I am using proxy to send requests, some of the guzzle(or cURL library) options are changed.

at final my question is that how can I send 100 async requests without using any proxies ?????

Upvotes: 0

Views: 663

Answers (1)

ArSeN
ArSeN

Reputation: 5248

The function curl_multi() which is used in the background by Guzzle afaik only allows a specific number of parallel requests at once. You might want to configure that number higher like this

curl_multi_setopt($curlHandle, CURLMOPT_MAXCONNECTS, 100)

and/or check out the other curl multi options here.

Now you might be wondering how to set that option on the curl handle that guzzle is using. According to this you can pass the curl options as an array to the Handlerstack.

Excerpt from linked docs:

use \GuzzleHttp\Client;
use \GuzzleHttp\HandlerStack;
use \GuzzleHttp\Handler\CurlMultiHandler;

$client = new Client(['handler' => HandlerStack::create(new CurlMultiHandler([
    'options' => [
        CURLMOPT_MAX_TOTAL_CONNECTIONS => 50,
        CURLMOPT_MAX_HOST_CONNECTIONS => 5,
    ]
]))]);

Note that the mentioned options in the example might be relevant in your case as well.

Upvotes: 1

Related Questions