shaelex
shaelex

Reputation: 261

Fill form with goutte/guzzle | symfony 4

I am trying to create a application which scrapes ads (eg. flat rental) and is able to create similar ad with just one click on exteral website. Scraping is done but I have no idea how to handle adding this advert. Authorization is pretty simple, so I made it by myself, but I have no clue how to get to this form, because before I can fill this form I have to choose some options with js buttons as shown on the gif below.

Part with authorization, login.

$crawler = $client->request('GET', 'https://www.gumtree.pl/login.html');
        dump($crawler->html());
        $form = $crawler->filter("form")->form();
        $crawler = $client->submit($form, array(
            'email' => 'email',
            'password' => 'password'
        ));

I am not attaching the another part with form, because it's not working anyway.

URL with form: https://www.gumtree.pl/post.html

How it looks: https://i.imgur.com/F00oLab.gifv

@edit

I made a request through my browser and next I tried to do the same request via symfony application, but it's still not working.

My chrome request:

request

My attempt with symfony:

$res = $client->get('https://www.gumtree.pl/post.html', [
            'auth' => [
                'username',
                'password'
            ],
            'form_params' => [
                'locationId' => '3200025',
                'categoryId' => '9073',
                'machineId' => 'xxxx',
                'DwellingForSaleBy' => 'ownr',
                'DwellingType' => 'flat',
                'AreaInMeters' => '50',
                'NumberRooms' => '3',
                'NumberBathrooms' => '10',
                'Parking' => 'grage',
                'Title' => 'titleeeeeeeee',
                'Description' => '<p>test desc</p>',
                'priceTypes' => 'FIXED',
                'Price' => '459999',
                'currencyValues' => 'PLN',
                'UserName' => 'username',
                'Email' => 'email',
            ]
        ]);

but the request in Symfony returns code 200, while it should return 302.

dump of response:

response

Any ideas how to deal with it?

Upvotes: 0

Views: 1263

Answers (1)

goto
goto

Reputation: 8164

The page you are scrapping is full of ajax, I think you should directly post the form data using Guzzle http://docs.guzzlephp.org/en/stable/quickstart.html#post-form-requests

$response = $client->request('POST', 'https://www.gumtree.pl/post.html', [
    'form_params' => [
        'field_name' => 'abc',
        'other_field' => '123',
        'nested_field' => [
            'nested' => 'hello'
        ]
    ]
]);

enter image description here

Upvotes: 1

Related Questions