CODEWITHSUNDEEP
python-3.xweb-scrapingscrapy
Rousblack
Rousblack

Reputation: 55

How can I get an element inside a class with scrapy using response.css

I'm trying to get value="3474636382675" from:

<input class="lst" value="3474636382675" title="Zoeken" autocomplete="off" id="sbhost" maxlength="2048" name="q" type="text">

I've tried

response.css(".lst >value").extract()

This one works but i'm getting everything back and i just need the value.

response.css(".lst").extract()

Upvotes: 1

Views: 3772

Answers (3)

sebostien
sebostien

Reputation: 111

With CSS you select the attribute you want like this:

response.css(".lst::attr(value)").extract()

You can read more about the selectors in Scrapy’s documentation

Upvotes: 5

akuiper
akuiper

Reputation: 214987

Not quite sure about css. But here is one from another SO answer. Alternatively try xpath:

response.xpath('//input[@class="lst"]/@value').extract()

or if you need only one value:

response.xpath('//input[@class="lst"]/@value').extract_first()

Upvotes: 0

kpie
kpie

Reputation: 11100

I use beautiful soup to parse html. Here's an example that grabs stock prices from yahoo finance.

import urllib.request
from bs4 import BeautifulSoup

def getPrice(tag):
    source = "https://finance.yahoo.com/quote/"+tag
    filehandle = urllib.request.urlopen(source)
    soup = BeautifulSoup(filehandle.read(), "html.parser")
    priceSpan = soup.findAll("span", { "class" : "Fz(36px)" })
    for k in priceSpan:
        return(k.getText())

def getDayChange(tag):
    source = "https://finance.yahoo.com/quote/"+tag
    filehandle = urllib.request.urlopen(source)
    soup = BeautifulSoup(filehandle.read(), "html.parser")
    priceSpan = soup.findAll("span", { "class" : "Fw(500)" })
    for k in priceSpan:
        return(k.getText())

https://gist.github.com/Krewn/0e624d35c396df63262dd42d74f2beb6

Upvotes: 0

Related Questions