CODEWITHSUNDEEP
imagepython-2.7downloadscrapy
zhen peng
zhen peng

Reputation: 31

How can I change the scrapy download image name in pipelines?

from __future__ import unicode_literals
import sys

from scrapy.pipelines.images import ImagesPipeline
from scrapy.exceptions import DropItem
from scrapy.http import Request
import os
reload(sys)
sys.setdefaultencoding('utf-8')

class TetePipeline(ImagesPipeline):
    def get_media_requests(self, item, info):
        for image_url in item['image_urls']:
            yield Request(image_url)

    def item_completed(self, results, item, info):
        item['image'] = []
        image_paths = [x['path'] for ok, x in results if ok]
        if not image_paths:
            raise DropItem('Items contains no images')
        item['image_paths'] = image_paths
        for i in item['image_paths']:
            item['image'].append(item['image_titles']+i[-8:]) 
        item['image_paths'] = item['image']

        return item

#

scrapy version :1.0 This is my code,It can download images,but the image names are the result of the image url SHA1 hash. I want to change the image name using custom name.in ths example is :item['image_titles']+i[-8:],int the scrapy shell the item['image_titles']+i[-8:] can be normal output,where is the reason?

Upvotes: 1

Views: 1021

Answers (1)

zhen peng
zhen peng

Reputation: 31

class TetePipeline(ImagesPipeline):
    def get_media_requests(self, item, info):
        for image_url in item['image_urls']:
           yield Request(image_url, meta={'item': item})

    def file_path(self, request, response=None, info=None):
        item = request.meta['item']
        image_guid = request.url.split('/')[-1]
        image_name = item['image_titles']+image_guid[-8:]
        return image_name

Change the file_path func, return the image_name, because the get_media_requests will download the image, item_completed has downloaded

Upvotes: 2

Related Questions