CODEWITHSUNDEEP
pythonphplaravelscreen-scraping
Rakii05
Rakii05

Reputation: 11

What tool for scraping in Laravel

I would like build a website needing scraping in other site, in Laravel 7. I have to fill forms and click. I saw that it's possible with python. The scraping will do itself every hours. I saw also Laravel dusk but it doesn't work in production environnement. What technologie you adviser me to use ? I don't know python so I will learn if it's more adapted.

Thanks

Upvotes: 1

Views: 1818

Answers (2)

Abed
Abed

Reputation: 61

There are a lot of options depending on what you want to do.

If you're working with a site that isn't an SPA, then you can simply use the included DOM package that comes with PHP, you can use curl to get the page HTML and parse it using the PHP DOM package.

However if you want to navigate the site and go through a scenario, then you'd best use something like Laravel Dusk, Puphpteer, Selenium.

In the past I've used Puphpteer before and it works great, here's a sample

$puppeteer = new Puppeteer();
$browser = $puppeteer->launch([
    'headless' => true,
    'args' => [
        '--no-sandbox',
        '--disable-setuid-sandbox'
    ]
]);

$page = $browser->newPage();
$page->goto('https://facebook.com');
$page->type('#username', 'email');
$page->type('#pass', 'password');
$page->click('button.btn__primary--large.from__button--floating');

This script opens facebook.com in a headless browser, and fills the fields with email and password and clicks on the button with the css selector of button.btn__primary--large.from__button--floating

Upvotes: 1

Igor Savinkin
Igor Savinkin

Reputation: 6267

fill forms and click

It means you have to do trivial POST requests. PHP curl is good at it:

$curl = curl_init('http://testing-ground.webscraping.pro/textlist');
curl_setopt($curl, CURLOPT_POST, true);
curl_setopt($curl, CURLOPT_POSTFIELDS, ['a' => 'abc']);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, TRUE);
$page = curl_exec($curl); 
if(curl_errno($curl)) // check for execution errors
{
    echo 'Scraper error: ' . curl_error($curl);
    exit;
} 
curl_close($curl);

For more details check here.

Upvotes: 0

Related Questions