Reputation: 261
I am trying to create a application which scrapes ads (eg. flat rental) and is able to create similar ad with just one click on exteral website. Scraping is done but I have no idea how to handle adding this advert. Authorization is pretty simple, so I made it by myself, but I have no clue how to get to this form, because before I can fill this form I have to choose some options with js buttons as shown on the gif below.
Part with authorization, login.
$crawler = $client->request('GET', 'https://www.gumtree.pl/login.html');
dump($crawler->html());
$form = $crawler->filter("form")->form();
$crawler = $client->submit($form, array(
'email' => 'email',
'password' => 'password'
));
I am not attaching the another part with form, because it's not working anyway.
URL with form: https://www.gumtree.pl/post.html
How it looks: https://i.imgur.com/F00oLab.gifv
@edit
I made a request through my browser and next I tried to do the same request via symfony application, but it's still not working.
My chrome request:
My attempt with symfony:
$res = $client->get('https://www.gumtree.pl/post.html', [
'auth' => [
'username',
'password'
],
'form_params' => [
'locationId' => '3200025',
'categoryId' => '9073',
'machineId' => 'xxxx',
'DwellingForSaleBy' => 'ownr',
'DwellingType' => 'flat',
'AreaInMeters' => '50',
'NumberRooms' => '3',
'NumberBathrooms' => '10',
'Parking' => 'grage',
'Title' => 'titleeeeeeeee',
'Description' => '<p>test desc</p>',
'priceTypes' => 'FIXED',
'Price' => '459999',
'currencyValues' => 'PLN',
'UserName' => 'username',
'Email' => 'email',
]
]);
but the request in Symfony returns code 200, while it should return 302.
dump of response:
Any ideas how to deal with it?
Upvotes: 0
Views: 1263
Reputation: 8164
The page you are scrapping is full of ajax, I think you should directly post the form data using Guzzle http://docs.guzzlephp.org/en/stable/quickstart.html#post-form-requests
$response = $client->request('POST', 'https://www.gumtree.pl/post.html', [
'form_params' => [
'field_name' => 'abc',
'other_field' => '123',
'nested_field' => [
'nested' => 'hello'
]
]
]);
Upvotes: 1