CODEWITHSUNDEEP
phpweb-scrapingcentos
Zeeshan Sultan
Zeeshan Sultan

Reputation: 26

PHP file_get_contents not working on CentOS

I am using file_get_contents to scrape a webpage. The code is very simple and works perfectly on windows, mac, and almost all major debian based linux distros. On CentOS the server sends 410 Gone instead.

Running on CentOS 7 64 bit with PHP 7.3

<?php

$x = file_get_contents('https://m.apkpure.com/facebook/com.facebook.katana/download'); 

echo $x;

On my MacOS On My Mac

On CentOS 7 On CentOS

Both OS' are up to date and are running PHP 7.3

Upvotes: 0

Views: 1584

Answers (2)

user13102176
user13102176

Reputation: 1

Execute this command:

sudo setsebool -P httpd_can_network_connect 1

Restart web service.

Upvotes: 0

exception_thrown
exception_thrown

Reputation: 586

The reason you are getting this error is because you need to pass a user agent (headers) when using https, or the server will reject the request.

To make things easier its better to use cURL, you can see if the function below fixes your issue. The reason it works on your windows and mac is because the server may be assigning a header/useragent to the request and linux is not.

A way to fix this on your linux box is by enabling allow_url_fopen in your php.ini

Try the function below. Without needing to enable allow_url_fopen

 function scraper($url) {
      $ch = curl_init();
      curl_setopt($ch, CURLOPT_HEADER, 0);
      curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
      curl_setopt($ch, CURLOPT_URL, $url);
      $data = curl_exec($ch);
      curl_close($ch);
      echo $data;
    }

    scraper("https://m.apkpure.com/facebook/com.facebook.katana/download");

Upvotes: 2

Related Questions