CODEWITHSUNDEEP
pythonscrapy
Evren Yurtesen
Evren Yurtesen

Reputation: 2349

Scrapy returning items within spider before first request/parse

I need to return some items from inside a spider before starting to parse requests. Because I need to make sure some parent items exist in database before parsing child items.

I now yield them from the parse method first thing, and this seem to work fine. But I was wondering if there is a better way to do this?

Upvotes: 0

Views: 103

Answers (1)

Gallaecio
Gallaecio

Reputation: 3847

Instead of yielding the items, write them into the database directly on the constructor of the pipeline where you add regular items to the database.

Upvotes: 1

Related Questions