CODEWITHSUNDEEP
scrapy
Bert Cafecito
Bert Cafecito

Reputation: 212

Different Item Pipelines for Spiders in One Project

I want to have one scrapy project that contains two spiders. I want each to have their item pipeline configure differently.

Any help will be appreciated, I tried to google a solution or any workaround. I have not found anything that made sense.

Upvotes: 1

Views: 145

Answers (1)

Georgiy
Georgiy

Reputation: 3561

You can implement this using spider custom settings:

import scrapy
from project.pipelines import Pipeline1, Pipeline2

class Spider1(scrapy.Spider):
    name = 'spider1'
    custom_settings = {
        'ITEM_PIPELINES':{
            Pipeline1: 100
        }
    ...

class Spider2(scrapy.Spider):
    name = 'spider2'
    custom_settings = {
        'ITEM_PIPELINES':{
            Pipeline2: 100
        }
    ...

Upvotes: 2

Related Questions