CODEWITHSUNDEEP
pythonparsingscrapy
GGA
GGA

Reputation: 385

Scrapy extract from every element of a list

I have this code in scrapy which creates a list of rows:

list_1 = single.xpath('//table[@class="day-table"]//tbody//tr').extract()

each element of the list has this example template

<img src="test"></img> <div class"class"></div>

how can i extract certain part of every element in the list and assigning to an item? My actual code is this

for u in list1:

item = classitem()
item['label1'] = map(unicode.strip, list1.xpath('(//tr/td[@class="day-table-seed"][1]/span/text())').extract())

i can't figure out how to make the extraction for 1 element at a time

Upvotes: 2

Views: 4000

Answers (1)

alecxe
alecxe

Reputation: 474171

In the loop, you need to use the u variable which would correspond to the next row matching your selector. Say you want the img element's src attribute to be written to an item field:

for u in list1:
    item = classitem()
    item['src'] = u.xpath('.//img/@src').extract_first()
    yield item

In Scrapy's terms list1 is a SelectorList instance, u is a Selector.

Also note that it is important to start your "inner" XPath expressions with a dot to make them work in a context of the current row.

Upvotes: 2

Related Questions