CODEWITHSUNDEEP
pythonseleniumweb-scrapingbeautifulsoupscrapy
4tire
4tire

Reputation: 11

Interactive scraping data from forms on website

I am new to the scrapping and with python in general. I have a task for myself. Actually I need to extract some data from forms. I have learned how to extract data from HTML elements, but this is little tricky for me. Task should look like this:
1. I need to login on website
2. Go to the specific URL
3. Then I need somehow to extract all data from form
Problem occurs because data of next dropdown button is shown when the previous one is choose. Here is a picture showing the site look and source code next to it.

Upvotes: 0

Views: 216

Answers (1)

4tire
4tire

Reputation: 11

    import scrapy

class AutoscoutSpider(scrapy.Spider):
    name = 'autoscout'
    allowed_domains = ['autoscout24.de']
    login_url = 'https://angebot.autoscout24.de/login?fromNavi=myAS24'
    start_urls = [login_url]

    def parse(self, response):
        token = response.css('input[name="__RequestVerificationToken"]::attr(value)').extract_first()
        podaci = {
        '__RequestVerificationToken' : token,
        'Username': '*********',
        'Password' : '********',
        }
        pass

This what I have done so far, I've just started making spider again. I'm using scrapy framework

Upvotes: 1

Related Questions