CODEWITHSUNDEEP
php
BlackFire27
BlackFire27

Reputation: 1550

Retrieving secured pages with curl

I scrape a few pages from google.. and one of the pages returns me this:

   File : C:\xampp\htdocs\PPC-Advert-System\Scrapper\ScrapperDBCRUD.php

Line: 29

SSL certificate problem, verify that the CA cert is OK. Details:\nerror:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

This is my curl code:

    private  function getPage($proxy, $url, $referer, $agent, $header, $timeout) {
        $ch = curl_init();
        curl_setopt($ch, CURLOPT_URL, $url);
        //curl_setopt($ch, CURLOPT_HEADER, $header);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_PROXY, $proxy);
        curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
        curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
         // curl_setopt($ch, CURLOPT_REFERER, $referer);
        //  curl_setopt($ch, CURLOPT_USERAGENT, $agent);
        $result['EXE'] = curl_exec($ch);
        $result['INF'] = curl_getinfo($ch);
        $result['ERR'] = curl_error($ch);
        if(curl_exec($ch) === false)
        {
            $result['ERR']=curl_error($ch);
        }
        curl_close($ch);
        return $result;
 }

I dont know what to add to the curl so it somehow verifies with the page I am trying to scrape. And also, I dont know if it may effect the scraping of ordinary pages that I scrape..whats the best way to cope with the issue?!?

I also get this error sometimes:

error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

Do both of those errors relate?

Upvotes: 0

Views: 104

Answers (1)

Benjamin Paap
Benjamin Paap

Reputation: 2759

If it is not necessary to verify the ssl connection you can add

curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false)

Upvotes: 2

Related Questions