CODEWITHSUNDEEP
pythonweb-scrapingscrapy
audiotec
audiotec

Reputation: 121

How to put a link in item with ItemLoader?

I would like to save a link into item with ItemLoader.

Basically I need to transform this code:

def parse(self, response): 
for casa in response.css('div.posting-card'):
    item = ZonapropItem()
    item['link'] = response.urljoin(casa.css('a.go-to-posting::attr(href)').get())
    yield item

Into:

def(parse, response):
for casa in response.css('div.posting-card'):
    loader = ItemLoader(ZonapropItem(),casa)
    loader.add_??? ('link', '???')
    yield loader.load_item()

I appreciate your answer!

Upvotes: 2

Views: 192

Answers (1)

Michael Savchenko
Michael Savchenko

Reputation: 1445

Hi again (: you could continue in your previous question.

There are two ways:

  • First one is to use loader.add_value() method with pre-computed value. Eg:
link = response.urljoin(casa.css('a.go-to-posting::attr(href)').get())
loader.add_value('link', link)

It's pretty straightforward.

  • The second one is to use MapCompose which will apply function from argument to every extracted item found by using CSS selector:
from scrapy.processors import MapCompose

loader.add_css('link', 'a.go-to-posting::attr(href)', MapCompose(response.urljoin))

Upvotes: 2

Related Questions