CODEWITHSUNDEEP
pythonxpathscrapy
OmGanesh
OmGanesh

Reputation: 1062

How to get scrapy xpath output in single item

i am extracting a particular xpath expression as:

#item["post"]=response.xpath("//div[contains(@id,'node-')]/div//p//text()").extract()

when i export the file in .csv format, the content is coming in separate line as:

Hi, Steps:,
1. Enabled calendar module.,
2. Enabled date api, date pop up, date views.

I want to export the post in a single line as follows:

Hi, Steps:, 1. Enabled calendar module., 2. Enabled date api, date pop up, date views.

Please help me out to solve the issue...

Upvotes: 1

Views: 433

Answers (2)

OmGanesh
OmGanesh

Reputation: 1062

Yes, the map() method worked fine for my scenario: item["post"]=map(unicode.strip,response.xpath("//div[contains(@id,'node-')]/div//p//text()").extract())

Upvotes: 0

GHajba
GHajba

Reputation: 3691

There are some options you could do.

First add multiple Fields to your item. But I guess this is not what you want.

The real cause of your problem is that when you use xpath('//...').extract() you get back a list of results. A solution for this would be to join the results together:

item["post"] = ' '.join(response.xpath("//div[contains(@id,'node-')]/div//p//text()").extract())

In your case it can be that even the elements have a line break so I would get rid of those new lines too if it is the case. For this you could use map(unicode.strip, ...).

Upvotes: 1

Related Questions