CODEWITHSUNDEEP
pythonscrapy
irokhes
irokhes

Reputation: 1703

Scrapy - Different pipeline per Item

I'm new with Scrapy and python so forgive my ignorance about this.

I need to store two different types of items in a database. For one of them, I need to do some extra queries before I do the insert. Is it possible to have a different pipelines based on the Item? If not, how can I differentiate which item is which when they get to the pipeline?

Upvotes: 2

Views: 1113

Answers (1)

hungneox
hungneox

Reputation: 9839

Basically you can discard the item that you don't wanna process in certain pipeline and vice ver sa. For example:

class ApplePipeLine(object):

    def process_item(self, item, spider):
        if not isinstance(item, Apple):
            return item
        # Do something with Apple
        return item


class OrangePipeLine(object):

    def process_item(self, item, spider):
        if not isinstance(item, Orange):
            return item
        # Do something with Orange
        return item

Upvotes: 4

Related Questions