CODEWITHSUNDEEP
pythonweb-scrapingscrapy
Luis Ramon Ramirez Rodriguez
Luis Ramon Ramirez Rodriguez

Reputation: 10393

Use scrapy in local data

I have used scrapinghub (A cloud service) to save the content of several pages in a CSV format, I have saved response.body In the rows of the CSV now I want to scrap this data stored locally I know I can use Bs4 but is posible to use scrapy too? I had try to use the Xpath but I get ''str object has no attribute xpath" how can I conver the string to the scrapy format again?

@Steve

I've try this 

df = pd.read_csv('items_bbb_2.csv')

data = df["Name"][0]

response = TextResponse(body=data, encoding='utf-8')

And get an error: "name 'TextResponse' is not defined"

Upvotes: 0

Views: 112

Answers (1)

Steve
Steve

Reputation: 976

You could try this

response = TextResponse(url=<url from csv or place holder>, body=<body from csv>, encoding='utf-8')

I've not tried this in this context but it should copy the body you supply into a Response object so that you can use xpath. I have used this with the body supplied by Selenium and it works fine.

Update: to get the defs for TextResponse, use:

from scrapy.http import TextResponse

The docs are here.

Upvotes: 2

Related Questions