CODEWITHSUNDEEP
javascriptphplaravelgoutte
ihya ainun Fikri
ihya ainun Fikri

Reputation: 65

How to Scrape "https://shopee.co.id/search?keyword=phone" with goutte library in laravel?

I want to get value from my code below, but the result is empty.

namespace App\Http\Controllers;

use Illuminate\Http\Request;
use Goutte\Client;

class ScrapeController extends Controller
{
    private $results = array();

    public function scraper(){
        $client = new Client();
        $raw = $client->request('GET', 'https://shopee.co.id/search?keyword=phone');
        $raw->filter('.col-xs-2-4')->each(function ($item) {
            $this->results[$item->filter('._1nHzH4')->text()] = $item->filter('_32hnQt')->text();
            
        });
        return $this->results;
    }
}

this code cant work. Anyone can give me the solution?

Upvotes: 0

Views: 258

Answers (2)

AH.Pooladvand
AH.Pooladvand

Reputation: 2059

This is happening because the content of the page is loading via javascript if you want to scrape the page you need to execute javascript first with a tool like phantomjs or use the php library https://github.com/jonnnnyw/php-phantomjs.
so in your case the steps will be:
1: execute javascript using phantomjs and get the raw HTML
2: pass the HTML to the goutte and then filter the data

Upvotes: 1

Ali Ahmad
Ali Ahmad

Reputation: 11

Use Guzzle HTTP client to make outgoing request from Laravel application.

private $results = array();

public function scraper(){
    $client = new Client();
    $guzzleClient = new GuzzleClient(array());
    $client->setClient($guzzleClient);
    //then make request
    $raw = $client->request('GET', 'https://shopee.co.id/search?keyword=phone');
    $raw->filter('.col-xs-2-4')->each(function ($item) {
        $this->results[$item->filter('._1nHzH4')->text()] = $item->filter('_32hnQt')->text();
        
    });
    return $this->results;
}

Upvotes: 1

Related Questions