CODEWITHSUNDEEP
djangoscrapy
Waheed Khan
Waheed Khan

Reputation: 1321

scrapy data not encoded correctly in python

I'm using scrapy for web crawling using python. While scraping I have some characters which are not encoded correctly like '\xa0','\x0259'. Any help how can I handle them in python?

Upvotes: 0

Views: 95

Answers (1)

ngutzmann
ngutzmann

Reputation: 51

You can use the unicode string type (http://docs.python.org/2/tutorial/introduction.html#unicode-strings) by prepending all instances of characters like these with u. For example u'\xa0' and u'\x0259'. The unicode-strings python docs also provide some other methods for encoding and decoding these strings and characters.

Upvotes: 1

Related Questions