CODEWITHSUNDEEP
phpweb-crawler
Instinct
Instinct

Reputation: 2251

Simple PHP web crawler returning simple HTML DOM error

I have a PHP script that returns links on a webpage. I am getting 500 internal error and this is what my server logs say. I let my friend try the same code on his server and it seems to run correctly. Can someone help me debug my problem? The warning says something about the wrapper is disabled. I checked line 1081 but I do not see allow_url_fopen.

PHP Warning: file_get_contents(): http:// wrapper is disabled in the server configuration by allow_url_fopen=0 in /hermes/bosweb/web066/b669/ipg.streamversetv/simple_html_dom.php on line 1081

PHP Warning: file_get_contents(http://www.dota2lounge.com/): failed to open stream: no suitable wrapper could be found in /hermes/bosweb/web066/b669/ipg.streamversetv/simple_html_dom.php on line 1081

PHP Fatal error: Call to a member function find() on a non-object in /hermes/bosweb/web066/b669/ipg.streamversetv/sim

<?php
 include_once('simple_html_dom.php');
 $target_url = 'http://www.dota2lounge.com/';
 $html = new simple_html_dom();
 $html->load_file($target_url);
  foreach($html->find(a) as $link){
    echo $link->href.'<br />';
  }
?>

Upvotes: 0

Views: 2478

Answers (2)

p.tamas
p.tamas

Reputation: 61

  1. Download latest simple_html_dom.php: LINK TO DOWNLOAD

  2. Open simple_html_dom.php in your favourite editor and add this code to the first couple of line(can be added right after <?php):

    function file_get_contents_curl($url) {

    $ch = curl_init();
    curl_setopt($ch, CURLOPT_AUTOREFERER, TRUE);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_URL, $url);     

    $data = curl_exec($ch);
    curl_close($ch);

    return $data; }

  3. Find line starting with function file_get_html($url..... for me it is line 71, but you can use search in your editor as well. (search for file_get_html)

  4. Edit this line(some lines below after function file_get_html):

    $contents = file_get_contents($url, $use_include_path, $context, $offset);

    to this:

    $contents = file_get_contents_curl($url);

  5. Instead of load_file, use file_get_html an it will work for you, without editing php.ini

Upvotes: 6

Jason
Jason

Reputation: 1962

You need to set the allow_url_fopen php setting to 1 to allow using fopen() with urls.

Reference: PHP: Runtime Configuration

Edit:
Also tracked down another thing, have you tried loading this way?

<?php
    include_once('simple_html_dom.php');

    $html = file_get_html('http://www.dota2lounge.com/');

    foreach($html->find('a') as $link)
    {
        echo $link->href.'<br />';
    }
?>

Upvotes: 1

Related Questions