CODEWITHSUNDEEP
phpweb-scrapinggoutte
zen
zen

Reputation: 1165

Setting proxy in Goutte

I've tried using Guzzle's docs to set proxy but it's not working. The official Github page for Goutte is pretty dead so can't find anything there.

Anyone know how to set a proxy?

This is what I've tried:

$client = new Client();
$client->setHeader('User-Agent', $user_agent);
$crawler = $client->request('GET', $request, ['proxy' => $proxy]);

Upvotes: 3

Views: 4764

Answers (5)

acollazo25
acollazo25

Reputation: 11

For recent versions use:

Goutte Client instance (which extends Symfony\Component\BrowserKit\HttpBrowser)

use Symfony\Component\HttpClient\HttpClient;
use Goutte\Client;

$client = new Client(HttpClient::create(['proxy' => 'http://xx.xx.xx.xx:80']));
...

Upvotes: 1

Mashpy Rahman
Mashpy Rahman

Reputation: 696

I have solved this problem =>

    $url = 'https://api.myip.com';
    $client = new \Goutte\Client;
    $client->setClient(new \GuzzleHttp\Client(['proxy' => 'http://xx.xx.xx.xx:8080']));
    $get_html = $client->request('GET', $url)->html();
    var_dump($get_html);

Upvotes: 3

Kiran Patel
Kiran Patel

Reputation: 379

You can directly use in Goutte or Guzzle Request 

$proxy = 'xx.xx.xx.xx:xxxx';

$goutte = new GoutteClient();
echo $goutte->request('GET', 'https://example.com/', ['proxy' => $proxy])->html();

Use Same method in Guzzle 

$Guzzle = new Client();
$GuzzleResponse = $Guzzle->request('GET', 'https://example.com/', ['proxy' => $proxy]);

Upvotes: 2

gandarrillas
gandarrillas

Reputation: 431

You can set a custom GuzzleClient and assign it to Goutte client. When Guzzle makes the request through Goutte uses the default config. That config is passed in the Guzzle construct.

$guzzle = new \GuzzleHttp\Client(['proxy' => 'http://192.168.1.1:8080']);
$goutte = new \Goutte\Client();
$goutte->setClient($guzzle);
$crawler = $goutte->request($method, $url);

Upvotes: 1

Ivan Gantsev
Ivan Gantsev

Reputation: 66

You thinking rigth, but in Goutte\Client::doRequest(), when create Guzzle client

$guzzleRequest = $this->getClient()->createRequest(
        $request->getMethod(),
        $request->getUri(),
        $headers,
        $body
);

options are not passed when create request object. So, if you want to use a proxy, then override the class Goutte\Client, the method doRequest(), and replace this code on

$guzzleRequest = $this->getClient()->createRequest(
        $request->getMethod(),
        $request->getUri(),
        $headers,
        $body,
        $request->getParameters()
);

Example overriding class:

<?php

namespace igancev\override;

class Client extends \Goutte\Client
{
    protected function doRequest($request)
    {
        $headers = array();
        foreach ($request->getServer() as $key => $val) {
            $key = implode('-', array_map('ucfirst', explode('-', strtolower(str_replace(array('_', 'HTTP-'), array('-', ''), $key)))));
            if (!isset($headers[$key])) {
                $headers[$key] = $val;
            }
        }

        $body = null;
        if (!in_array($request->getMethod(), array('GET','HEAD'))) {
            if (null !== $request->getContent()) {
                $body = $request->getContent();
            } else {
                $body = $request->getParameters();
            }
        }

        $guzzleRequest = $this->getClient()->createRequest(
            $request->getMethod(),
            $request->getUri(),
            $headers,
            $body,
            $request->getParameters()
        );

        foreach ($this->headers as $name => $value) {
            $guzzleRequest->setHeader($name, $value);
        }

        if ($this->auth !== null) {
            $guzzleRequest->setAuth(
                $this->auth['user'],
                $this->auth['password'],
                $this->auth['type']
            );
        }

        foreach ($this->getCookieJar()->allRawValues($request->getUri()) as $name => $value) {
            $guzzleRequest->addCookie($name, $value);
        }

        if ('POST' == $request->getMethod() || 'PUT' == $request->getMethod()) {
            $this->addPostFiles($guzzleRequest, $request->getFiles());
        }

        $guzzleRequest->getParams()->set('redirect.disable', true);
        $curlOptions = $guzzleRequest->getCurlOptions();

        if (!$curlOptions->hasKey(CURLOPT_TIMEOUT)) {
            $curlOptions->set(CURLOPT_TIMEOUT, 30);
        }

        // Let BrowserKit handle redirects
        try {
            $response = $guzzleRequest->send();
        } catch (CurlException $e) {
            if (!strpos($e->getMessage(), 'redirects')) {
                throw $e;
            }

            $response = $e->getResponse();
        } catch (BadResponseException $e) {
            $response = $e->getResponse();
        }

        return $this->createResponse($response);
    }
}

And try send request

$client = new \igancev\override\Client();
$proxy = 'http://149.56.85.17:8080'; // free proxy example
$crawler = $client->request('GET', $request, ['proxy' => $proxy]);

Upvotes: 2

Related Questions