The function file_path parameter in imagepipiline responds. When I return str directly, I can successfully download the image. If response.meta.get('file_name') is used, the download will fail. It is also a string. Why can't the variable be used?
item.py -
class ImageItem(scrapy.Item):
file_name = scrapy.Field()
url = scrapy.Field()
test.py -
class TestSpider(scrapy.Spider):
name = 'test'
allowed_domains = ['www.baidu.com']
start_urls = ['http://www.baidu.com/']
def parse(self, response):
image_item['file_name'] = 'yoxo.jpg'
image_item['url'] = 'https://zmtimg.dfcfw.com/529d2d0f7a57c9135eb956b9afa1810a.em/art'
yield image_item
pipelines.py -
class ArticleImagePipeline(ImagesPipeline):
def file_path(self, request, response=None, info=None):
try:
print('response.file_name:{}'.format(response.meta.get('file_name')))
print(type(response.meta.get('file_name')))
print(response.meta.get('file_name'))
except Exception as e:
pass
return 'hello.jpg'
def get_media_requests(self, item, info):
meta = {
'file_name': item['file_name']
}
yield Request(url=item['url'], meta=meta)
when i'm running it .it successful!The results are as follows:
response.file_name:yoxo.jpg
<class 'str'>
yoxo.jpg
response.file_name:yoxo.jpg
<class 'str'>
yoxo.jpg
But when I use a variable as the return value, it failed.
def file_path(self, request, response=None, info=None):
try:
print('response.file_name:{}'.format(response.meta.get('file_name')))
print(type(response.meta.get('file_name')))
print(response.meta.get('file_name'))
except Exception as e:
pass
return 'hello.jpg'
result:
2018-12-13 22:11:39 [scrapy.core.scraper] WARNING: Dropped: Image Downloaded Failed
{'file_name': 'yoxo.jpg',
'url': 'https://zmtimg.dfcfw.com/529d2d0f7a57c9135eb956b9afa1810a.em/art'}
I tried it with the following code. The result is empty, why? The same is the first response normal return, written outside the try is empty
try:
# aa = response.meta.get('file_name')
print('response.file_name:{}'.format(response.meta.get('file_name')))
print(type(response.meta.get('file_name')))
print(response.meta.get('file_name'))
except Exception as e:
logging.error(response)
logging.error(e)
pass
print(response.meta.get('file_name','yep'))
return 'hello.jpg'
result:
2018-12-13 22:15:31 [root] ERROR: None
2018-12-13 22:15:31 [root] ERROR: 'NoneType' object has no attribute 'meta'
2018-12-13 22:15:31 [scrapy.core.scraper] WARNING: Dropped: Image Downloaded Failed
{'file_name': 'yoxo.jpg',
'url': 'https://zmtimg.dfcfw.com/529d2d0f7a57c9135eb956b9afa1810a.em/art'}
why?I couldn't understand 。 Many thanks for any help~