CODEWITHSUNDEEP
pythonunit-testingscrapy
NewbAndroider
NewbAndroider

Reputation: 5

Writing unit test for scrapy, yielding item

So I'm trying to get into a habit of writing unit tests for my project in scrapy, but the thing is, I'm not sure how to go about this. For example, in my self.parse(), I always yield the item rather than return the item, so whenever I call a self.parse(), it returns a generator object. What I tried doing what generating a fake response as shown here:

Scrapy Unit Testing

It seems to work if you are returning an item, but what about an item that is being yielded? How do you test the items that are being yielded?

Upvotes: 0

Views: 1347

Answers (1)

Granitosaurus
Granitosaurus

Reputation: 21436

Simply consume generator into a list:

expected = MyItem({'foo': 'bar'})
item = list(spider.parse(response))
assert item == expected

Upvotes: 1

Related Questions